Simplify your Drupal Deployments

Kevin

Kevin


Aug 12th 2020 in Développement web

As a developer, I'm a big fan of having an automated deployment process. It's really into the web development philosophy build once, and deploy everywhere.

When I prepare a release or just having a development demo, I don't want to struggle with deployment. I don't want to even think about it.
I want a solution wrote once, bound with a Continuous Deployment System (TravisCI, CircleCI, GithubActions or any other ...), easy to maintain, reliable and proven to be working well by the Community.

In the past, I have used various home-made shell scripts to perform those tasks.

For the last 4 years, I have been converted to Capistrano. With it, I'm able to deploy with a single command cap staging deploy. When I'm ready to launch in production, it's just cap production deploy.

At Antistatique, we contributed and maintained the well known Capdrupal Gem, a library providing tasks which are useful for deploying & managing Drupal 9, Drupal 8 & Drupal 7 projects with Capistrano. We use it on all our Drupal projects.

By using Capdrupal, I bring stability and consistency to my release management process, and it's a critical step on the way to Continuous Delivery.

Well, I hope those lines convince you of all the benefits of Capistrano. Otherwise, I'm sure it will speak by itself later in this article.

:coffee: What else ?


Why Should I Automate, Anyway?

Development takes time. You need to understand the problem you need to solve, plan the architecture, decide on the best tools, write the code and resolve the bugs. All this can take hours, days, weeks, months or even years. At the end of this cycle is the deployment process, which can be not only time demanding but also frustrating.

Sneak peak of CI/CD pipelines


Therefore, eliminating these troubles can go a long way in freeing up time for the actual software development. This is what automated deployment does; it ensures that you only have to deal with setting up the deployment process once. After that, the computer handles future deployments, so you can place greater focus on producing quality code.

Why Capistrano?

Deploying code to Drupal installations has always been a bit of a struggle. Although there are a few Drupal modules or Bash scripts that may help in deployments, there hasn't really been a simple Drupal deployment process. It's about time that this process becomes a lot easier.

Simply put, Capistrano is a tool for automating tasks on one or more remote servers. It executes commands in parallel on all targeted machines, and provides a mechanism for rolling back changes across multiple machines.

Let's say we have four Drupal environments, for a site we are working on: local, development, staging and production. Each of these environments would use separate databases, completely different file system and would be accessible from different hostnames.

For each of these environments we should be able to checkout code (from a specific branch), backup the database, activate maintenance mode, synchronize the latest Drupal configuration changes and run the necessary update (drush updatedb). Then, if everything works, publish the new application otherwise rollback.

At one of my previous jobs, we attempted to handle deployments with a very long and confusing PHP script. It was very complex, contained lots of developers' individual environment details, and certainly was not the most eloquent solution. Obviously, it was slow and not documented.
Worse, those scripts were copied/past on every project. Therefore, the scripts evolved between projects without being updated in a single location.

Me dealing with those old PHP recipies


By using Capistrano, Git, Drush & Capdrupal we can create a normalized, simplified, well documented, working and reliable deployment process.
A release management process used by many and proven to work well for most use cases.

On the other hand, considering that the Symfony Community is using Capistrano as one of its main Deployment Tool, I think the Drupal Community should also consider it as a strong choice.

Goals and Assumptions

Now, before we dig in I have a few items to cover:

  • You're already familiar with how to use Drush,
  • You're using Drupal 8/9 or higher,

Second, we should outline some goals for deployment automation:

  • Minimize the likelihood of bugs or downtime,
  • During the deployment process, the current online environment should stay untouched,
  • During the deployment the current project should be put in maintenance mode,
  • A deployment should be indicated as failed if the failed step in the script cannot be successfully addressed with automation,
  • The state of the system at a failed deployment should be auditable at a future time, even after a rollback is complete,
  • The rollback is optimized for restoring the entire system to a previous state,

Now, let's look at the Capistrano integration. It's important to keep in mind as you read through those lines that they are intended to achieve the goals a Drupal project deployment. There are cases where these scripts are unfit, so let's consider these lines a starting point from which you can make adjustments for those special cases.

Steps-by-Steps

First, you'll need to install Capistrano, as well as the specific Drupal gem Capdrupal:

gem install capistrano
gem install capdrupal

Configurations

Go to your project root directory and launch Capistrano.

cd path/to/your/drupal/project/
cap install

Capistrano will create the following skeleton:

.
├── Capfile
├── config
│   └── deploy.rb
│   └── deploy
│       └── production.rb
│       └── staging.rb
├── lib
│   └── capistrano
│        └── tasks

You will now update the created Capfile and require 2 new dependencies:

# Composer is needed to install drush on the server.
require 'capistrano/composer'

# Drupal Tasks.
require 'capdrupal'

Then, go to config/deploy.rb to set the parameters of your project. First you have to define the general information about the user, server and the app itself.

set :application, 'application-name'
set :repo_url, 'git@github.com:company/application.git'

server 'ssh.example.org', user: 'username', roles: %w{app db web}

The specific Drupal information

set :install_composer, true
set :install_drush, true

set :app_path, 'web'
set :config_path, 'config/sync'

# Setup the backup before/after failing strategy.
set :backup_path, 'backups'
set :keep_backups, 5

# Link file settings.php
set :linked_files, fetch(:linked_files, []).push("#{fetch(:app_path)}/sites/default/settings.php", "drush/drush.yml")

# Link dirs files and private-files
set :linked_dirs, fetch(:linked_dirs, []).push("#{fetch(:app_path)}/sites/default/files")

Finally, setup the deployment process to use the proper Drupal 8/9 strategy. This is what does the real work. If you aren't a ruby programmer, don't panic; neither am I. Hopefully, this is general-purpose enough that it will work for you out of the box.

Additionally, individual tasks can be overridden in deploy.rb or development.rb – so once you have a Capfile you like the same file can be reused across all your projects.

namespace :deploy do
  # Ensure everything is ready to deploy.
  after "deploy:check:directories", "drupal:db:backup:check"

  # Backup the database before starting a deployment and rollback on fail.
  # after :updated, "drupal:db:backup"
  # before :failed, "drupal:db:rollback"
  # before :cleanup, "drupal:db:backup:cleanup"

  # Set the maintenance Mode on your Drupal online project when deploying.
  after :updated, "drupal:maintenance:on"

  # Must updatedb before import configurations, E.g. when composer install new
  # version of Drupal and need updatedb scheme before importing new config.
  # This is executed without raise on error, because sometimes we need to do drush config-import before updatedb.
  after :updated, "drupal:updatedb:silence"

  # Remove the cache after the database update
  after :updated, "drupal:cache:clear"
  after :updated, "drupal:config:import"

  # Update the database after configurations has been imported.
  after :updated, "drupal:updatedb"

  # Clear your Drupal cache.
  after :updated, "drupal:cache:clear"

  # Disable the maintence on the Drupal project.
  after :updated, "drupal:maintenance:off"

  # Ensure permissions are properly set.
  after :updated, "drupal:permissions:recommended"
  after :updated, "drupal:permissions:writable_shared"

  # Fix the release permission (due to Drupal restrictive permission)
  # before deleting old releases.
  before :cleanup, "drupal:permissions:cleanup"
end

You may now configure your staging.rb and production.rb strategy, as you will certainly deploy on different environments.

# staging.example.org
set :deploy_to, '/var/www/staging.example.org'

# set a branch for this release
set :branch, 'dev'

# Map composer and drush commands
# NOTE: If stage have different deploy_to
# you have to copy those line for each <stage_name>.rb
# See https://github.com/capistrano/composer/issues/22
SSHKit.config.command_map[:composer] = -> { shared_path.join('composer.phar') }
SSHKit.config.command_map[:drush] = -> { release_path.join('vendor/bin/drush') }

Then update the production.rb file

# www.example.org
set :deploy_to, '/var/www/example.org'

# set a branch for this release
set :branch, 'main'

# Map composer and drush commands
# NOTE: If stage have different deploy_to
# you have to copy those line for each <stage_name>.rb
# See https://github.com/capistrano/composer/issues/22
SSHKit.config.command_map[:composer] = -> { shared_path.join('composer.phar') }
SSHKit.config.command_map[:drush] = -> { release_path.join('vendor/bin/drush') }

Usage

You're now ready to run cap deploy:setup.

So, after configuration comes action, you're now ready to run cap deploy:setup !

cap [staging|production] composer:install_executable
cap [staging|production] deploy:setup

Capistrano creates directories and a symlink to the targeted server. The shared directory contains all shared files of your app who don't need to be change. releases contains the different releases of your app with a number define in deploy.rb and finally current is the symlink who target the right release.

example.org
├── current -> /var/www/example.org/releases/20130527070530
├── releases
│   ├── 20130527065508
│   ├── 20130527065907
│   └── 20130527070530
└── shared

This task only needs to be ran once, prior to the first deployment. It will create the necessary directories on your server, which will ultimately look something like the tree above. (Until your first deployment, there may not be anything in the releases directory, nor a link from current). The web server document root in this example is /var/www/example.org/current.

Now, every time you want to deploy your app, just run:

cap [staging|production] deploy

You should then be able to proceed as you would usually, you may want to familiarize yourself with the truncated list of tasks, you can get a full list with:

cap -T

Lock your dependencies

The Gemfile and its generated Gemfile.lock allows you to specify the versions of the dependencies that your deployment process needs, while remembering all the exact versions of third-party code that your application used when it last worked correctly.

This is a mandatory step to ensure consistency between environments

Append the following content to your Gemfile:

source 'https://rubygems.org'

group :development do
  gem 'capistrano', '~> 3.14'
  gem 'capistrano-composer', '~> 0.0.6'
  gem 'capdrupal', '~> 3.0.0'
end

Then execute the bundle install command that install the needed gems and generate a Gemfile.lock. (this works like a composer.json and composer.lock files, but you need bundler gem installed).

When using bundler, all commands needs to be prefixed with bundle exec, such as bundle exec cap staging deploy.


Sources

For the most curious of you, here are some sources of additional information that inspired the creation of this article.

Github (07 August, 2020). Capdrupal
Seen on https://github.com/antistatique/capdrupal

Chris Svajlenka (13 March, 2012). Simplify your WordPress Deployments
Seen on https://www.metaltoad.com/.../simplify...

Dylan Tack (6 November, 2009). Capistrano: Drupal deployments made easy
Seen on https://www.metaltoad.com/.../capistrano-drupal...

Billy Davies (4 Juillet, 2013). Professional deployment of websites using Capistrano
Seen on https://www.zodiacmedia.co.uk/.../professional-deployment...

Jochen Verdeyen (15 December, 2015). Professional deployment of websites using Capistrano
Seen on https://www.slideshare.net/JochenVerdeyen/deploying-drupal...

Joseph Purcell (12 March, 2020). How to Automate Drupal Deployments with Rollback
Seen on https://www.bounteous.com/.../.../automate-drupal-deployments

Shankar (1 November, 2018). Automate Drupal deployment using Infrastructure as Code
Seen on https://opensenselabs.com/.../drupal-infrastructure-code

Conferences

Michiel Rook (6 October, 2017). I Deploy On Fridays
Seen on https://www.youtube.com/watch?v=_i7t56daSLs

Jochen Verdeyen (13 September, 2013). Deploying Drupal with Capistrano
Seen on https://www.youtube.com/watch?v=nDrxmmqC64g

Oliver Davies (1 November, 2019). Deploying PHP applications with Ansible, Ansible Vault and Ansistrano
Seen on https://www.youtube.com/watch?v=7yhEmKRWZ5Y

Capistrano alternatives

Deployer
This is another native PHP rewrite of Capistrano, with some ready recipes for Drupal.

Ansistrano
An Ansible role that allows you to configure a powerful deploy via YAML files.

Magallanes
This Capistrano-like deployment tool is built in PHP, and may be easier for PHP developers to extend for their needs.

Fabric
This Python-based library provides a basic suite of operations for executing local or remote shell commands and uploading/downloading files.