Migrate your Drupal 7 site to Drupal 9

I recently worked on the revamp project La Fonciere - which is a straightforward migration of a Drupal 7 website to Drupal 9. During that project I realised how few step-by-step use-case articles exist about Migrating a Drupal 7 project to Drupal 9.

And that’s a shame ! Indeed, Drupal 7 will only be supported until November 2022, meaning more and more projects will be required to be migrated from D7 to D9.

In this blogpost, I will explain the how-to process I used to migrate La Fonciere, let’s dig in !One more thing before starting, when I use the word “migration”, I mean the process to re-create and move data from an existing source (Eg. an old Drupal 7 project) to the new location (Eg. the new Drupal 9 project). This process includes the creation of different content types, taxonomy terms, files and users, which are a big part of what's important in a website. Obviously, during a migration we still have to recreate all the configuration from scratch, which is good to get around legacy websites and revise what was done. The client needs may have evolved and we have to adapt the project architecture.

First step - Create the migration module

When your brand new D9 site is installed, we can start working on the migration. First, we'll need to add some dependencies to make the migration easier for us. You'll need drupal/migrate_plus and drupal/migrate_tools. These modules will add plugins on top of the ones available with the Migrate module from the Core.

When this is done, we can create a custom module that will held all the migration business logic. Create a new folder in web/modules/custom with the name your_project_name_migration and add the file your_project_name_migration.info.yml.

It should look like this :

## filename : web/modules/custom/your_project_name_migration/your_project_name_migration.info.yml


name: 'your_project_name_migration'
type: module
description: 'Store migration from previous D7 website'
core_version_requirement: ^8.8 || ^9
package: your_project_name
dependencies:
  - drupal:migrate_plus
  - drupal:migrate_tools
  - drupal:migrate
  - drupal:migrate_drupal
  - drupal:node
  - drupal:taxonomy

You may add dependencies for what you will migrate, like drupal:user or drupal:files if you need both.

When this is done, you can create a directory that will hold all the migrations files, let's call it migrations. All the migration files will be created there.

The first file to create is the group for the whole migration process, so we can run one command and all the migration will be run at once.

Create the file migrate_plus.migration_group.your_project_name_migration.yml and append this content :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration_group.your_project_name_migration.yml

id: your_project_name_migration

label: 'Import from previous site (Drupal 7)'

description: 'Migrations from the previous Drupal 7 website.'

# Short description of the type of source, e.g. "Drupal 6" or "WordPress".
source_type: 'Drupal 7'

# Configure the source database
shared_configuration:
  source:
    # replace this by a databases key in settings.php or drush arguments.
    key: migrate

# By default, configuration entities (like this migration) are not automatically
# removed when the migration which installed them is uninstalled. To have your
# migrations uninstalled with your migration module, add an enforced dependency
# on your module.
dependencies:
  enforced:
    module:
      - your_project_name_migration

We now have a migration group named your_project_name_migration, we can start working on the migrations of Files (including both Images and Documents).

I prefer starting with Files, since most content type depend on Files because of referenced images or documents in the Content Type.

Migrate Files

Create a file migrate_plus.migration.files.yml. We'll go through all the important pieces of configuration.

Migration identification

The first thing is to add the config to identify the migration and add it to the migration group that was created before.

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.files.yml

id: files
label: Files
migration_group: your_project_name_migration
migration_tags:
  - files

id : The id of the migration.
label : Name of the migration.
migration_group : Specify the group so that the migration will be run when the group is run.
migration_tags : List of tags if we want to run migration by tags instead of the whole group.

Migration source

We need to specify where we can get the information from the source database.

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.files.yml

source:
  plugin: d7_file
  key: migrate
  scheme: public
  constants:
    uid_root: 1
    source_base_path: 'public://migrations'

plugin : We use the source plugin provided by the core Migrate module, d7_file.
scheme : We only want to get the public files.
constants : We can specify variables that will be later used in the migration file.
uid_root: If we don't migrate the users or the users are not important, we need to specify an uid and 1 is the admin which is always created by Drupal.
source_base_path : To migrate the files, we need to have the files in the local filesystem or the migration will fail. And we can specify here the path to the folder with all the files and use it later.

Migration process

This is the main part of the migration. It's in this part that we specify where the source data should be associated with in the new structure. Here is an example for files :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.files.yml

process:
  fid: fid
  filename: filename
  source_full_path:
    -
      plugin: concat
      delimiter: /
      source:
        - constants/source_base_path
        - filepath
  uri:
    plugin: file_copy
    source:
      - '@source_full_path'
      - uri
  filemime: filemime
  status: status
  # Drupal 7 didn't keep track of the file's creation or update time -- all it
  # had was the vague "timestamp" column. So we'll use it for both.
  created: timestamp
  changed: timestamp
  uid: constants/uid_root

fid : This is the identifier for a file and it didn't change in D9, so we just get the information from the source.
filename : Self-explanatory.
source_full_path : It is a variable that will be later used in the uri. We use the plugin concat to concatenate the constant that was set beforehand and the filepath that was set in the D7 database.
uri : We use the plugin file_copy to get the entire file path and add it to the new database.
filememe : Self-explanatory.
status : Get the status (published or unpublished)
created / changed : As said in the comment, D7 only had a timestamp unlike D9 that stores more accurate information.
uid : As said before, get the constant to specify the admin as the creator.

Migration destination

The last thing is to specify where all this processed data will be saved to

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.files.yml

destination:
  plugin: entity:file

This part is to specify that we want to migrate the data processed to file entities.

With that, the migration for the files should be done and we can start with other content. We can continue with the taxonomy terms.

Migrate taxonomy terms

For each taxonomy you want to migrate, you'll need to create a migration file, but the file themselves will be very similar and the differences will be :

The source
The destination
The fields

Let's see how we created these migrations.

Migration source file

Unlike the files, we didn't use a provided source plugin for taxonomies, but we needed to create a source plugin ourselves. To do so, we're gonna create a class in the module we created in src/Plugin/migrate/source. We decided on this naming <NameOfYourTaxonomy>Term.php.

Here is an example of a generic source plugin class you can use :

// filename : /web/modules/custom/your_project_name_migration/src/Plugin/migrate/source/ArticleCategoryTerm.php

<?php

namespace Drupal\your_project_name_migration\Plugin\migrate\source;

use Drupal\migrate\Plugin\migrate\source\SqlBase;

/**
 * Migration class for article category taxonomy.
 *
 * @MigrateSource(
 *   id = "article_category_term",
 *   source_module = "your_project_name_migration"
 * )
 */
class ArticleCategoryTerm extends SqlBase {

  /**
   * {@inheritdoc}
   */
  public function query() {
    $query = $this->select('taxonomy_term_data', 'td')
      ->fields('td', [
        'tid',
        'vid',
        'language',
        'name',
        'description',
      ]);

    $query->condition('td.vid', 1);

    return $query;
  }

  public function fields() {
    return [
      'tid',
      'vid',
      'language',
      'name',
      'description'
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function getIds() {
    return [
      'tid' => [
        'type' => 'integer',
        'alias' => 'td',
      ],
    ];
  }
}

Some explanation :

The Annotation @MigrateSource : This is how we can specify that this is a source plugin for a migration and we can use it.

In the class, 3 methods are required :

query() : This is where you can build the query to the source database and get the information.
fields() : List of fields to get from the source.
getIds() : The ids for the source, here, it's only tid

With that done, we can create the migration configuration file. We'll use an article category taxonomy as an example, but it can be whatever you want. Create the file migrate_plus.migration.article_category.yml in the migrations directory.

Migration identification

Similar to the files, we need to add an identification to the migration :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article_category.yml

id: article_category
label: Article category
migration_group: your_project_name_migration
migration_tags:
  - article

Everything is the same as the files migration.

Migration source

This is where we'll specify to use the source plugin we created :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article_category.yml

source:
  plugin: article_category_term
  key: migrate

The key will be explained later, the important part is the plugin where we use the source plugin we created before.

Migration process

Again, we'll need to specify the process, or mapping, of files between the source and the destination.

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article_category.yml

process:
  langcode:
    plugin: get
    source: langcode
  vid:
    plugin: default_value
    default_value: article_category
  name:
    plugin: get
    source: name
  description:
    plugin: get
    source: description

langcode : We use the plugin get, which is the standard one and you don't need to specify it, but we prefer to be a bit more verbose and make sure that it's understood more easily.
vid : We use the plugin default_value which is self-explanatory and we set the vid to the machine_name of the new taxonomy term in the D9 website
name : Get the name from the source
description : Get the description from the source

Migration destination

Again, we need to specify the destination entity in D9 :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article_category.yml

destination:
  plugin: entity:taxonomy_term
  default_bundle: article_category

The only difference from the files migration is the default_bundle, we need to specify it so the migration knows which taxonomy term should the data be migrated to.

Migrate nodes

Now that the files and the taxonomy terms were created, we can focus on the nodes or the content types. We'll continue with the article example since it's a content type that is created by default by Drupal and is present in a lot of websites.

We'll also explain how to migrate translated content, which is vital in many websites. We'll need to create 2 files :

migrate_plus.migration.article.yml
migrate_plus.migration.i18n.yml

The first one is for the migration in the original language, the second for the other languages.

Migration identification - Original language

Very similar than the other migrations :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article.yml

id: article
label: Article - Node
migration_group: your_project_name_migration
migration_tags:
  - article

Migration source - Original language

Like the files, we have a source plugin provided by the core Migrate module :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article.yml

source:
  plugin: d7_node
  key: migrate
  node_type: article
  constants:
    uid_root: 1

plugin : We use the d7_node module which provides everything to get the information from a D7 node.
node_type : The bundle name of the node in the D7 site

Migration dependencies

Since we'll have references to files and taxonomies that should be migrated before, we add dependencies to make sure that the migration is run after them :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article.yml

migration_dependencies:
  required:
    - article_category
    - files

Migration process - Original language

The most important points are :

nid
langcode
body (or field you chose for the rich text)
category field
Image or file field(s)
Links

Here is an example for all of these fields :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article.yml

process:
  nid: tnid
  langcode:
    plugin: default_value
    source: language
    default_value: 'und'
  body/value:
    - plugin: extract
      field: body/value
      index:
        - 0
  body/format: 'full_html'
  field_category:
    plugin: migration_lookup
    migration: article_category
    source: field_category
  field_external_link:
    plugin: field_link
    uri_scheme: 'https://'
    source: field_external_link
  field_cover_image:
    plugin: sub_process
    source: field_cover_image
    process:
      target_id: fid
      alt: alt
      title: title
      width: width
      height: height
  field_attachment:
    plugin: sub_process
    source: field_attachment
    process:
      target_id: fid
      alt: alt
      title: title
      width: width
      height: height

nid : We just need to make sure to get the tnid and not the nid.
langcode : We can combine the default_value plugin with the source, so we can put a default value if no value was found in the source data.
body : It is separated between the value and the format. For the value, we need to extract the value from the body in D7, that's why the plugin extract is used. The format is put directly and you can use whichever one you'd like. Just be careful that you won't use a format that doesn't have everything the D7 site had.
field_category : For references to other content (user, taxonomy term or nodes), we need to use the plugin migration_lookup where you'll specify from which migration the data comes from. In this case, the data will come from the article_category migration.
field_external_link : For links, we need to use the plugin field_link which will process the link. In this case, we want to handle an external link, so we specify a prefix so Drupal knows it's external.
field_cover_image / field_attachment : Both are similar since an image (if not from the Media module) is a file entity. For files, we need the sub_process plugin so we can specify the target_id being the id of the file itself as well as other informations.

And the last thing is the destination :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.article.yml

destination:
  plugin: 'entity:node'
  default_bundle: article

After creating the migration for the original language, we can create the migration for the translations.

Migration identification - Translations

Again, similar to other migrations :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.i18n.yml

id: article_i18n
label: Article - Node
migration_group: your_project_name_migration
migration_tags:
  - article

Migration source - Translations

This is where it differs from the original language, we're gonna use a specific source plugin :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.i18n.yml

source:
  plugin: d7_node_entity_translation
  key: migrate
  node_type: article

Migration destination - Translations

We also need to specify in the destination that the content is translated content :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.i18n.yml

destination:
  plugin: 'entity:node'
  translations: true
  destination_module: content_translation

Migration process - Translations

It is the same as the article in the original language article, but we only need to process the translated fields.

Migration dependencies - Translations

We need to depend on the original language migration :

## filename : web/modules/custom/your_project_name_migration/migrations/migrate_plus.migration.i18n.yml

migration_dependencies:
  required:
    - article

Now that all the migrations files are created, we can finalise the setup to run the migrations.

Second step - Database connection setup

Now that the migrations files are written, we want to test what we did. And for that, we need to setup the connection to the database where the source resides. We'll need to change the settings.php file which is located in web/sites/default. At the end of the file, add this :

// filename : web/sites/default/settings.php

$databases['migrate']['default'] = array (
  'database' => 'drupal_seven',
  'username' => 'drupal',
  'password' => 'drupal',
  'host' => 'db',
  'port' => '3306',
  'namespace' => 'Drupal\\Core\\Database\\Driver\\mysql',
  'driver' => 'mysql',
);

You just need to enter the correct information for your database, save the file and you should be good to go.

Third step - Run the migration !

Here we are, finally, after this hard work, we can run the migrations and make sure that everything works as expected.

First thing, enable the migration module created from the Drupal admin or with the drush command :

drush en your_project_name_migration

It should prompt you to accept also enabling the other migrate modules that we put as dependencies, answer yes to that.

When this is done, we can run the command to run our migration group :

drush mim --group="your_project_name_migration"

Watch as the migrations are run one after the other and if everything is correct, the command should finish without errors and you can go to your Drupal site and check that everything was migrated as expected.

Conclusion

While the documentation can be a bit scarce, Drupal has a lot of modules that will help you to migrate your data from D7 to a new D9 (or D8, it works exactly the same).

Here are all the different process plugins available from the core Migrate module and Migrate plus module.

Hope that this module was helpful ! We are always happy to get feedback and any correction if we explained something wrong.