Gatsby incremental build for self-hosted Drupal environments

Why is a Gatsby incremental build required?

Every time you run a Gatsby build, the following steps are processed internally:

Delete CSS and HTML files at the very beginning
Source data from an endpoint
Create pages and queries
Build JS/CSS production files
Build HTML files

For sites with a large amount of content, this process can be quiet time-consuming. For instance, you make modifications to only 2 nodes in a Drupal site that has 20,000 nodes. However when you run the Gatsby build it will consider changes for all the 20,000 nodes and process the above the steps.

Solution

For sites that are hosted on the Gatsby Cloud, the solution to the problem is Gatbsy Cloud builds feature.

But for self-hosted Drupal environments, since this is cannot be the solution Gatsby provides us with environmental variables that helps us to reduce the build time by possessing the .cache and .public directories between the builds.

So every time you run a Gatsby build with the environment variable flag. It compares the page data from the previous build and creates a list of page directories that are finally passed to the build process. Which means it will only trigger build for the changed nodes.

The difference between the build time is quite visible. For our Drupal site that has no more than 100 nodes, there was a difference of 50 seconds in the consequent builds which I think is pretty huge.

The first build will trigger build for all the pages.

Gatsby Incremental build for self-hosted Drupal environments

On the second build, only the pages with content change are updated.

How to implement a Gatsby incremental build?

1. Upgrade your Gatsby to the latest version


npm install gatsby@latest

2. Add a flag to your build command in package.json


"scripts": {

  "develop": "gatsby develop",

  "build": "GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true                  
gatsby build --log-pages"

},

or you can simply use the environment variable in your terminal every time you run your build command.

To get this working with Netlify you need to add the netlify-plugin-gatsby-cache to your netlify.toml config file. You can follow this blog to see how it is implemented.

Things to remember

GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES=true flag must be used every time you trigger a new build, otherwise, Gatsby will not be able to persist the cache between the builds and you will not get the desired outcome.
Any code changes or change to a static query will trigger a new build. The environment variable only takes care of content changes.
Also, if you add or delete any new content/node, a new build is triggered.

For more in-depth information about how this works, you can follow the pull request details here.

P.S - In case you have any queries, you may mention them in the comments section.Incremental build functionality is now a part of the Gatsby core. Although an experimental feature, it is pretty significant and a much-awaited one. Let's understand why the Gatsby incremental build is required for self-hosted Drupal environments and how to implement it.Note: This post is also applicable for other backends and environments.