a drupal migration story, part i: a simple migration

Migrating content from one system to another is a common task in web development. In the Drupal world, the Migrate API has been the go-to tool for this task. Together with modules like Migration Tools that provide a number of enhancements over Drupal Core’s, finding the plugins to fit your needs is even easier.

In this example, we’ll explore how to migrate content from a JSON file into a Drupal 10 site. We’ll use the Migrate API and the Migrate Tools module to accomplish this. We’re going with JSON because it’s a common format for data exchange, supported by many systems, and is easy to work with.

Before We Begin

Before we get started, we’ll need to make sure we have a few things in place, namely a Drupal 10 site and the Migrate Tools module installed and enabled. We’ll also need to have Drush installed to help us scaffold the custom module we’ll use to handle the migration. As developed here, I have Drupal 10.2, Migrate Tools 6.0, and Drush 12.4.

We’ll also want to use Drupal’s Media Library module to handle the images on our side. This will allow us manage the images separately from the content, reference them from multiple places, and as an added bonus, minimize duplication.

The JSON file

The data we’ll be migrating comes courtesy of an old Drupal 7 site that, for our purposed, needs picked part and rebuild from scratch in Drupal 10. Hence, the data is ‘massaged’ to return a unique node id, title, summary in plain text, body in HTML, a single hero image object with the source, alt text, and credit, created and updated timestamps, and a boolean published status.

[
  {
    "nid": 1,
    "title": "A Sample News Article",
    "summary": "This is a summary of the news article.",
    "body": "<p>This is the body of the news article.</p>",
    "hero": {
      "src": "source.jpg",
      "alt": "A sample hero image",
      "credit": "Picture Taker"
    },
    "created": "2024-01-04T00:00:00.000Z",
    "updated": "2024-01-04T00:00:00.000Z",
    "published": true
  }
]

The Module

We’ll create a custom module to handle the migration. We’ll use Drush to scaffold the module and provide some boilerplate code.

drush generate module
  Module name: ten mig
  Module machine name [ten_mig]:
  Module description: a simple json-based migration module
  Package [Custom]:
  Dependencies (comma separated): migrate_tools,media,media_library
  Would you like to create module file? [No]:
  Would you like to create install file? [No]:
  Would you like to create README.md file? [No]:Yes
  The following directories and files have been created or updated:
  –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
  • /var/www/web/modules/custom/ten_mig/README.md
  • /var/www/web/modules/custom/ten_mig/ten_mig.info.yml

Once the module is created and enabled, we’ll need to create a yaml migration configuration files that will define the source, process, and destination of the migration. Since the news articles’ images will be referenced from the site’s media library, we’ll start by creating migrations to handle the images first.

The Migration Sauce

The Media library store images in folders named by the media’s create date year and month. We’ll try to emulate this structure in order to keep with this convention.

The first migration file will copy the images from their source folder to the media library. The second migration file will create a media entity for each image.

In both cases, we’ll use Drush to generate the yaml migration configuration files.

drush generate yml:migration

This is a sample of the prompt to generate the first image migration file.

Welcome to migration generator!
–––––––––––––––––––––––––––––––––
Module machine name: ten_mig
Migration label: news images
Migration ID [ten_mig_news_images]: ten_mig_news_images
Source plugin ID: url
Destination plugin ID: entity:file
The following directories and files have been created or updated:
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
• /var/www/web/modules/custom/ten_mig/migration/ten_mig_news_images.yml

Drush generate the file with some boilerplate code that needs cleaned up and modified to fit our needs. The repository for this example contains all the migration files that are heavily commented to explain the various parts.

How To

To run the migration, we’ll use Drush. Since the article migration depends on the media migration, which in turn depends on the image migration, the order of operation is key. We can run each migration individually or run all at once. The former is useful for debugging and testing, and the latter is useful for production — when you want to save time and know that your data migrates without errors.

drush migrate:import ten_mig_news_images,ten_mig_news_media,ten_mig_news

Outro

Hopefully, this example has given you a good starting point for migrating content from a JSON file into a Drupal 10 site. The Migrate API is a powerful tool that can handle a wide variety of data sources and destinations. It’s worth exploring it’s many plugins to find what best fits your needs. The Migrate Plus also has a number of plugins that can be useful in more complex data transformations not covered by the Migrate API.

In the next post, we’ll look at data transformations and custom plugins that can handle more complex data.

• #drupal
• #migration