Track Experiments Automatically
Track experiments, capturing every detail including performance, generated files, logs, and source code.
Analyze, Compare, Optimize
Learn from each experiment to optimize your model in less time — apply your own insights or use Auto ML.
Get Started Fast — No Code Changes
Use your current training scripts without modification — there's no need to adopt another framework or library.
How does Guild AI help?
Guild helps you train better models in less time. Effective machine learning is a function of systematic experimentation — one experiment leads to another as you deepen your understanding. The faster and more effective you can apply experiments, the sooner you'll complete your work.
Who uses Guild AI?
Guild AI is used by machine learning engineers and researchers to run, track, and compare experiments. Each experiment yields valuable information that is captured and used to inform next steps. Scientists and developers leverage their experiment results to build deeper intuition, troubleshoot issues, and automate model architecture and hyperparameter optimization.
How is Guild AI different?
Guild is cross platform and framework independent — you can train and capture experiments in literally any language using any library. Guild runs your unmodified code, which means you get to use the libraries you want. Guild doesn't require databases or other infrastructure to management experiments — it's simple and easy to use. This frees you to focus on what's important: building state-of-the-art models and machine learning apps.
When you run your training scripts with Guild, you get experiment management automatically. Use the results to make informed decisions about your models.
Guild makes it easy to optimize hyperparameters and model architecture. With a single command, you can apply state-of-the-art algorithms to your training scripts.
Guild makes it easy to recreate your experiments. By systematically running your scripts and capturing their output, Guild lets others run and compare results.
Guild provides a suite of visualization, comparison, and diffing tools for studying and comparing your experiment results.
Guild has a powerful workflow feature that lets you optimize over a series of operations, letting you apply Auto ML to true end-to-end learning.
Remote training and backups
Guild lets you train remotely — e.g. on powerful GPU servers — as well as backup and restore your runs.
Run your training script with Guild to generate an experiment
- Guild runs your script directly — no need to change your code
- Captures files, metrics, output, and logs as a unique experiment
Run multiple trials for a set of choices (Grid Search)
[0.1,0.2,0.3]is a list of three values — Guild runs a trial for each
- Runs trials over the Cartesian product of all values — i.e. performs a grid search
Run trials for a range (Random Search)
[0.1:0.3]is a range of values — from minimum to maximum
- In this example Guild runs 10 trials, selecting values at random over a uniform distribution
Find the best hyperparameters using Bayesian optimization
--optimizerto minimize (or maximize) an objective
- Guild supports the latest Bayesian optimizers including gaussian process, decision trees, and gradient boosted trees
Analyze and Compare Results
After running experiments, use
guild compare to
launch an interactive spreadsheet-like application to explore,
sort, and compare results.
- Spreadsheet-like application to compare experiment results
- Flexible display — customize what you see from the command line
- Mark best results for export or use in other trials
Export to CSV or JSON
- Generate CSV or JSON files containing experiment details
- Use with other tools and programs to analyze and visualize results
guild view to start a browser-based
application to explore and compare experiment results.
- Explore and compare trial results in a browser
- Run as a shared server to support group collaboration
- Seamless integration with TensorBoard to view trial scalars (e.g. loss, accuracy, etc.) and other logged events
TensorBoard is used to study and compare trial metrics and
generated output. Use
guild tensorboard to quickly
and easily start TensorBoard for your experiments.
- Compare loss, accuracy and other training metrics across runs
- Guild keeps TensorBoard up-to-date as you run new trials
- Support for generating TensorBoard logs automatically from training output
Diff changes across experiments
guild diff to compare two trials.
- Compare every detail: hyperparameters, source code, logs, command, environment and any generated file
- Use to answer, “What changed between these two runs that may have influenced this result?”
- Customize using the diff tools of your choice
Step 1: Add a Guild file
to your project
A Guild file is a simple text file named
that describes the operations for your project. Guild files
support a range of features that automate reproducibility
— just it to your project root and you're done!
Step 2: Share your code
By adding a Guild file to your project, you make it easy for colleagues and other researchers to recreate your experiments. Simply share your project code through GitHub or your favorite version control system.
Step 3: Colleagues use Guild
to recreate experiments
After cloning your project repo, others can recreate your
experiments using the
guild run command. Guild
takes care of everything needed to train your model and captures
the results for easy comparison.
Guild file — instructions for recreating an experiment
train.py: flags: epochs: 3 activation: default: relu choices: - relu - sigmoid num_dense_layers: 1 num_dense_nodes: 16 learning_rate: 1e-5 requires: - url: http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz - url: http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz - url: http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz - url: http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
guild.yml located in project root directory
flags section defines hyperparameters. Guild
uses the default values if not otherwise specified by the
user. This makes it easy for someone to recreate an experiment
by simply running
requires section defines a list of files that
are needed by the operation. Guild automatically downloads
required files and makes them available to training script.
Steps to recreate an experiment
- Run command in project root — i.e. the directory
- Guild downloads the required files and
train.pyusing the default hyperparameter values defined in the Guild file
- Training results are automatically captured and available for comparison to published results
It just takes a moment to install Guild and get started with a simple experiment. From there you can learn about more advanced features.Get Started
Explore the features
Not convinced that Guild is right for you? Spend a few more minutes browsing its features.Guild AI features
Browse Guild AI docs
If you're interested in a complete picture of Guild AI, start by browsing its comprehensives documentation.Browse the docs