Docs Navigation

Caching between builds

Installing application dependencies can take a significant amount of time, so Semaphore caches them to make your build time shorter.

Semaphore caches certain directories automatically, and some only if the the relevant language is selected in Project Settings.

Package manager specific sections, including examples for caching dependencies for nested projects:

In all cases, only top-level directories are cached, residing in $SEMAPHORE_PROJECT_DIR.

Automatically cached directories

The following directories are always cached, regardless of the language selected for the project.

.semaphore-cache
.bundle
vendor/bundle
node_modules

The generic .semaphore-cache directory is located in /home/runner/your-app/.semaphore-cache. The full path is available in the $SEMAPHORE_CACHE_DIR environment variable. In case you need to download a lot of dependencies, it's a good idea to keep them in this cache directory.

Caching additional directories

Often, it's beneficial to cache generic directories as well, besides the project dependencies. For this, utilizing symlinks is a good approach.

In the following example, the _build and deps directories are cached for mix, a popular Elixir build tool.

mkdir -p $SEMAPHORE_CACHE_DIR/{_build,deps}
ln -s $SEMAPHORE_CACHE_DIR/deps $SEMAPHORE_PROJECT_DIR/deps
ln -s $SEMAPHORE_CACHE_DIR/_build $SEMAPHORE_PROJECT_DIR/_build

Now when mix compile populates deps and _build in the project directory ($SEMAPHORE_PROJECT_DIR), the artifacts go directly to Semaphore's cache directory.

Docker layer caching

Docker images can be cached by using Docker's built-in --cache-from functionality, which is described in more detail on this page.

Bundler

Bundler is popular outside of Ruby projects as well, and this is the reason its directories are cached automatically, regardless of the selected project language.

Semaphore assumes that the gems are installed to the default .bundle, or to vendor/bundle with bundle install --path vendor/bundle.

As only the top-level .bundle and vendor/bundle directories are cached automatically, nested projects should install their dependencies with the following command, to utilize caching:

cd nested-project; bundle install --path $SEMAPHORE_CACHE_DIR/bundle; cd -

NPM

The usage of Node packages is common in other languages other than JavaScript as well, and node_modules is cached automatically.

As only the top-level node_modules directory is cached automatically, nested projects should install their dependencies with the following command, to utilize caching:

mkdir -p $SEMAPHORE_CACHE_DIR/node_modules; rm -rf $SEMAPHORE_PROJECT_DIR/nested-project/node_modules
ln -fs $SEMAPHORE_CACHE_DIR/node_modules $SEMAPHORE_PROJECT_DIR/nested-project/node_modules
cd nested-project; npm install; cd -

NPM doesn't support changing the local installation path, so setting a path with --cache will not work. A workaround like above is needed.

Yarn

Yarn dependencies are cached only when JavaScript is selected in Project Settings.

As only the top-level node_modules directory is cached automatically, nested projects should install their dependencies with the following command, to utilize caching:

cd nested-project; yarn install --modules-folder $SEMAPHORE_CACHE_DIR/nested-project-modules; cd -

Composer

Composer dependencies are cached only if PHP is selected in Project Settings. Semaphore assumes that the Composer cache is in vendor or in ~/.composer.

As only the top-level vendor directory is cached automatically, nested projects should install their dependencies with the following command, to utilize caching:

cd nested-project; COMPOSER_CACHE_DIR=$SEMAPHORE_CACHE_DIR/composer-cache composer install; cd -

Gradle

The ~/.gradle/caches directory is cached only when Java is selected in Project Settings.

Maven

The ~/.m2 directory is cached when either Java or Clojure is selected in Project Settings.

sbt

The ~/.sbt directory is cached only when Scala is selected in Project Settings.

Apache Ivy

The ~/.ivy2 directory is cached only when Scala is selected in Project Settings.

Hex

The ~/.hex directory is cached only when Elixir is selected in Project Settings.

If we are not caching packages of your favourite language, please let us know how you are using them and we will expand our system.

Semaphore Docs are open source — Edit on GitHub

Newsletter

Occasional lightweight product and blog updates. Unsubscribe at any time.

2009-2017 © Rendered Text. All rights reserved. Terms of Service, Privacy policy, Security.