Track Experiments Automatically

Track experiments, capturing every detail including performance, generated files, logs, and source code.

Analyze, Compare, Optimize

Learn from each experiment to optimize your model in less time — apply your own insights or use Auto ML.

Get Started Fast — No Code Changes

Use your current training scripts without modification — there's no need to adopt another framework or library.



How does Guild AI help?

Guild helps you train better models in less time. Effective machine learning is a function of systematic experimentation — one experiment leads to another as you deepen your understanding. The faster and more effective you can apply experiments, the sooner you'll complete your work.

Who uses Guild AI?

Guild AI is used by machine learning engineers and researchers to run, track, and compare experiments. Each experiment yields valuable information that is captured and used to inform next steps. Scientists and developers leverage their experiment results to build deeper intuition, troubleshoot issues, and automate model architecture and hyperparameter optimization.

How is Guild AI different?

Guild is cross platform and framework independent — you can train and capture experiments in literally any language using any library. Guild runs your unmodified code, which means you get to use the libraries you want. Guild doesn't require databases or other infrastructure to management experiments — it's simple and easy to use. This frees you to focus on what's important: building state-of-the-art models and machine learning apps.


Features

Experiment management

When you run your training scripts with Guild, you get experiment management automatically. Use the results to make informed decisions about your models.

Auto ML

Guild makes it easy to optimize hyperparameters and model architecture. With a single command, you can apply state-of-the-art algorithms to your training scripts.

Reproducibility

Guild makes it easy to recreate your experiments. By systematically running your scripts and capturing their output, Guild lets others run and compare results.

Analytics

Guild provides a suite of visualization, comparison, and diffing tools for studying and comparing your experiment results.

End-to-end learning

Guild has a powerful workflow feature that lets you optimize over a series of operations, letting you apply Auto ML to true end-to-end learning.

Remote training and backups

Guild lets you train remotely — e.g. on powerful GPU servers — as well as backup and restore your runs.


Run experiments

Run your training script with Guild to generate an experiment

$ guild run train.py learning-rate=0.1
  • Guild runs your script directly — no need to change your code
  • Captures files, metrics, output, and logs as a unique experiment

Get started with experiments

$ guild run train.py learning-rate=[0.1,0.2,0.3]
  • [0.1,0.2,0.3] is a list of three values — Guild runs a trial for each
  • Runs trials over the Cartesian product of all values — i.e. performs a grid search

Get started with Grid Search

$ guild run train.py learning-rate=[0.1:0.3] --max-trials 10
  • [0.1:0.3] is a range of values — from minimum to maximum
  • In this example Guild runs 10 trials, selecting values at random over a uniform distribution

Get started with Random Search

Find the best hyperparameters using Bayesian optimization

$ guild run train.py x=[-2.0:2.0] --optimizer bayesian
  • Use --optimizer to minimize (or maximize) an objective
  • Guild supports the latest Bayesian optimizers including gaussian process, decision trees, and gradient boosted trees

Guide to Hyperparameter Optimization


Analyze and Compare Results

Compare experiments

$ guild compare

After running experiments, use guild compare to launch an interactive spreadsheet-like application to explore, sort, and compare results.

Interactive compare application
  • Spreadsheet-like application to compare experiment results
  • Flexible display — customize what you see from the command line
  • Mark best results for export or use in other trials

Guide to comparing runs

Export to CSV or JSON

$ guild compare --csv > results.csv
  • Generate CSV or JSON files containing experiment details
  • Use with other tools and programs to analyze and visualize results

Guide to exporting runs

Integrated visualizers

$ guild view

Use guild view to start a browser-based application to explore and compare experiment results.

Files associated with a trial

Trial output
  • Explore and compare trial results in a browser
  • Run as a shared server to support group collaboration
  • Seamless integration with TensorBoard to view trial scalars (e.g. loss, accuracy, etc.) and other logged events
$ guild tensorboard

TensorBoard is used to study and compare trial metrics and generated output. Use guild tensorboard to quickly and easily start TensorBoard for your experiments.

Compare experiment results in TensorBoard

View and compare model architecture
  • Compare loss, accuracy and other training metrics across runs
  • Guild keeps TensorBoard up-to-date as you run new trials
  • Support for generating TensorBoard logs automatically from training output

Diff changes across experiments

$ guild diff

Use guild diff to compare two trials.

Detailed diff of two runs (trials)

Diff source code changes between two runs
  • Compare every detail: hyperparameters, source code, logs, command, environment and any generated file
  • Use to answer, “What changed between these two runs that may have influenced this result?”
  • Customize using the diff tools of your choice

Reproducibility

Step 1: Add a Guild file
to your project

A Guild file is a simple text file named guild.yml that describes the operations for your project. Guild files support a range of features that automate reproducibility — just it to your project root and you're done!

Step 2: Share your code
with colleagues

By adding a Guild file to your project, you make it easy for colleagues and other researchers to recreate your experiments. Simply share your project code through GitHub or your favorite version control system.

Step 3: Colleagues use Guild
to recreate experiments

After cloning your project repo, others can recreate your experiments using the guild run command. Guild takes care of everything needed to train your model and captures the results for easy comparison.

Guild file — instructions for recreating an experiment

train.py:
  flags:
    epochs: 3
    activation:
      default: relu
      choices:
        - relu
        - sigmoid
    num_dense_layers: 1
    num_dense_nodes: 16
    learning_rate: 1e-5
  requires:
   - url: http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
   - url: http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
   - url: http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
   - url: http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz

guild.yml located in project root directory

The flags section defines hyperparameters. Guild uses the default values if not otherwise specified by the user. This makes it easy for someone to recreate an experiment by simply running guild run.

The requires section defines a list of files that are needed by the operation. Guild automatically downloads required files and makes them available to training script.

Steps to recreate an experiment

$ guild run
  • Run command in project root — i.e. the directory containing guild.yml (above)
  • Guild downloads the required files and runs train.py using the default hyperparameter values defined in the Guild file
  • Training results are automatically captured and available for comparison to published results
Guild uses the information in the Guild file to replicate the experiment, keeping things simple for new users.

Get started with reproducibility


Next Steps

Quick Start

It just takes a moment to install Guild and get started with a simple experiment. From there you can learn about more advanced features.

Get Started

Explore the features

Not convinced that Guild is right for you? Spend a few more minutes browsing its features.

Guild AI features

Browse Guild AI docs

If you're interested in a complete picture of Guild AI, start by browsing its comprehensives documentation.

Browse the docs