Almost every Drupal build of any size eventually has to swallow content it didn't author: a CSV export from a legacy CMS, a JSON feed from a publishing partner, a decade of articles trapped in a system nobody wants to keep running. The temptation is to write a one-off script that loops over the rows and calls Node::create(). It works once. Then someone asks you to run it again next week against an updated feed, and you discover you've created five thousand duplicate nodes. The Migrate API exists precisely to keep that from happening, and learning to model an import as a migration — rather than a script — is one of the higher-leverage skills a Drupal developer can build.
The core idea is the map table. Every migration keys each source record on a stable identifier and records, in a dedicated table, which destination entity that record produced. On the next run, Migrate consults the map: a record it has seen before updates the existing node; a record it hasn't creates a new one. That single mechanism is what makes a migration idempotent — safe to run a hundred times — and it's why the unique source id deserves more thought than anything else in the pipeline. If your feed has a real primary key, use it. If it doesn't, derive a deterministic one; never key on something that can drift, like a title.
A clean migration has three moving parts: a source plugin that yields rows, a process pipeline that transforms each field, and a destination plugin that writes the entity. The source can be anything iterable — a database, a CSV, a JSON file — and writing a small custom source plugin is far less work than people expect. Here's the heart of a file-backed JSON source:
protected function initializeIterator(): \Iterator {
$raw = file_get_contents($this->resolveSourcePath());
$decoded = json_decode($raw, TRUE);
if (isset($decoded['articles'])) {
$decoded = $decoded['articles'];
}
return new \ArrayIterator(array_values($decoded));
}
public function getIds(): array {
return ['external_id' => ['type' => 'string', 'max_length' => 191]];
}The getIds() declaration is the whole game: it tells Migrate which field is the map key. Everything downstream — idempotency, rollback, "update existing" — flows from getting it right.
The process pipeline is where raw data becomes clean content. Legacy text is almost always dirty: doubled whitespace, stray control characters, HTML where you wanted plain text, free-form category labels that should be taxonomy terms. Resist the urge to scatter that cleanup across a dozen inline transforms. Instead, push the logic into a service and call it from a thin process plugin. A normalizer service that collapses whitespace, derives a summary and reading time, and folds data-migration and Data Migration into the same term name becomes the single source of truth — reused by the migration and by an entity presave hook, and unit-testable without booting Drupal at all. Resolving those topic labels into real terms with entity_generate means your taxonomy is built as a side effect of the import, consistently, instead of by hand afterward.
Two operational habits separate a migration that survives production from one that doesn't. First, model the content properly before you import a single row: define the content type, fields and vocabulary as exported configuration so the model is versioned, reviewable, and identical across environments. An import into a content model that only exists in one developer's database is a time bomb. Second, make the run observable. Wrap it in a Drush command that reports source rows, imported counts and unprocessed rows, and surface the same numbers on an admin page so a content editor — not just an engineer — can confirm a run worked. A migration you can't watch is a migration you can't trust.
These instincts generalize well beyond content. The same discipline — stable keys, idempotent re-runs, a reversible path back — is what makes any large data move safe, whether you're running zero-downtime schema migrations at scale or moving user accounts atomically between systems. And the habit of pushing logic into a small, testable service rather than scattering it through callbacks pays off everywhere; it's the same move that makes hunting N+1 queries systematically tractable instead of a guessing game. Treat your imports as software, not scripts, and the third run is as calm as the first.
Run it
The full source is on GitHub — github.com/tachyurgy/drupal-migrate-module (MIT). Cloned fresh, it runs with its base toolchain and nothing else. Here is an actual run:
External Content Migrate — verification output
Generated: 2026-06-06 21:10 PDT
PHP: PHP 8.5.7 (cli) (built: Jun 2 2026 20:59:56) (NTS)
NOTE: This module is real, idiomatic Drupal 10/11 code. Full EXECUTION
(drush ecm:import, the migration, the report page, Kernel tests) requires a
bootstrapped Drupal site, which is not available in this environment. What is
verified here is static correctness: every PHP file passes 'php -l', and every
YAML/JSON/XML file parses. The code is written to drop into a Drupal site as-is.
==================================================================
1. php -l — syntax check on EVERY PHP file
==================================================================
No syntax errors detected in ./src/Commands/ExternalContentMigrateCommands.php
No syntax errors detected in ./src/Controller/MigrationReportController.php
No syntax errors detected in ./src/Plugin/migrate/process/NormalizeText.php
No syntax errors detected in ./src/Plugin/migrate/source/ExternalArticleJson.php
No syntax errors detected in ./src/Service/ContentNormalizer.php
No syntax errors detected in ./src/Service/ContentNormalizerInterface.php
No syntax errors detected in ./src/Service/MigrationReportService.php
No syntax errors detected in ./tests/src/Unit/ContentNormalizerTest.php
No syntax errors detected in ./tests/src/Unit/NormalizeTextProcessTest.php
==================================================================
2. YAML validity — parsed with python3 yaml.safe_load
==================================================================
OK ./config/install/core.entity_form_display.node.external_article.default.yml
OK ./config/install/core.entity_view_display.node.external_article.default.yml
OK ./config/install/field.field.node.external_article.body.yml
OK ./config/install/field.field.node.external_article.field_author.yml
OK ./config/install/field.field.node.external_article.field_reading_minutes.yml
OK ./config/install/field.field.node.external_article.field_source_url.yml
OK ./config/install/field.field.node.external_article.field_topic.yml
OK ./config/install/field.storage.node.field_author.yml
OK ./config/install/field.storage.node.field_reading_minutes.yml
OK ./config/install/field.storage.node.field_source_url.yml
OK ./config/install/field.storage.node.field_topic.yml
OK ./config/install/migrate_plus.migration_group.external_content.yml
OK ./config/install/node.type.external_article.yml
OK ./config/install/taxonomy.vocabulary.article_topics.yml
OK ./drush.services.yml
OK ./external_content_migrate.info.yml
OK ./external_content_migrate.links.menu.yml
OK ./external_content_migrate.permissions.yml
OK ./external_content_migrate.routing.yml
OK ./external_content_migrate.services.yml
OK ./migrations/external_articles.yml
==================================================================
3. JSON validity — parsed with python3 json.load