Guild home is a directory that Guild AI uses to store various files.
Every Guild command is associated with a Guild home. If Guild home does not exist, Guild will automatically create it.
By default, Guild home is one of two locations depending on whether or not a command is run in a virtual environment.
If the command is not run in a virtual environment, Guild home is
~/.guild by default. If the command is run in a virtual environment,
Guild home is
$VIRTUAL_ENV is the
virtual environment directory.
Guild home may be set explicitly by either defining the environment
GUILD_HOME or by using the
‑H option when running a
Guild home is structured as follows:
- Guild homeE.g. ~/.guild or $VIRTUAL_ENV/.guild
- cache Caches used to optimize various Guild functions
- resources Cached resources
- runs Indexed run data
- runs Active runs
- trash Deleted runs
Active runs (i.e. non-deleted runs) are stored in
$GUILD_HOME/runs. Each run is stored in a subdirectory with the same
name as the run ID.
Run directories contain all files associated with a run, including:
- Resource source links
- Output generated during the run
- Guild metadata
Guild metadata is stored in a subdirectory named
.guild in each run
- Run directory
- .guild Guild metadata
- output Output generated by the run OS process
- output.index Timestamp and stream information associated with run output
- attrs Run attributes
- cmd Run OS process command
- deps Run dependencies (required resources)
- env Run OS process environment
- exit_status Run OS process exit status
- _extra_NAME Extra config for NAME, defined for the model operation
- flags Flag values used for the run
- opref Reference to the operation associated with the run
- started Started timestamp
- stopped Stopped timestamp
When Guild resolves resources, it uses
for two purposes:
- Location for downloading URL resource sources
- Location for unpacking archives
Resources are stored in directories that are named using hashes of the corresponding resource source URI, which is an internal representation of a resource source location. If a resource exists in the cache, it is used rather than downloading or unpacking the resource again.
Guild does not remove items from the resource cache.
You may clear the resource cache by deleting
GUILD_HOME/cache/resources. Guild commands will continue to function
normally and the cache will be repopulated as needed.
Resource cache and virtual environments
By default, Guild reuses the resource cache in
~/.guild/cache/resources (the user-level resource cache) for all
virtual environments. This means that cached resources available
within virtual environments without having to re-download
files. However, it also means that virtual environments can modify the
user-level resource cache.
If you want to isolate a virtual environment from the user-level cache, run the following command after you first activate the environment:
guild init --local-resource-cache
Guild maintains an index of run data in
index lets Guild lookup values such as training accuracy and loss
quickly when performing comparisons.
Guild updates the runs index lazily as it discovers new runs or new data associated with a run.
You may clear the runs cache by deleting
GUILD_HOME/cache/runs. Guild commands will continue to function
normally and runs will be indexed again as needed.