Musings of a PC

Thoughts about Windows, TV and technology in general

Automating Site Building

As I mentioned in Building a Website That Costs Pennies to Operate, the initial technical design of the infrastructure had the website layout defined in a private git repository and the content in a public git repository.

The private git server used was Atlassian BitBucket – the self-hosted version, not the cloud version. Although Linaro’s IT Services department is very much an AWS customer, we had already deployed BitBucket as an in-house private git service so it seemed to make more sense to use that rather that choose to pay an additional fee for an alternative means of hosting private repositories like CodeCommit or GitHub.

So what to do about the build automation? An option would have been to look at CodeBuild but, as Linaro manages a number of Open Source projects, we benefit from Atlassian’s very kind support of the Open Source community, which meant we could use Atlassian Bamboo on the same server hosting BitBucket and it wouldn’t cost us any more money.

For each of the websites we build, there is a build plan. The plans are largely identical to each other and go through the following steps, essentially emulating what a human would do:

  • Check out the source code repositories
  • Merge the content into a single directory
  • Ensure that Jekyll and any required gems are installed
  • Build the site
  • Upload the site to the appropriate S3 bucket
  • Invalidate the CloudFront cache

Each of these is a separate task within the build plan and Bamboo halts the build process whenever a task fails.

There isn’t anything particularly magical about any of the above – it is what CI/CD systems are all about. I’m just sharing the basic details of the approach that was taken.

Most of the tasks in the build plan are what Bamboo calls a script task, where it executes a script. The script can either be written inline within the task or you can point Bamboo at a file on the server and it runs that. In order to keep the build plans as identical as possible to each other, most of the script tasks run files rather than using inline scripting. This minimises the duplication of scripting across the plans and greatly reduces the administrative overhead of changing the scripts when new functionality is needed or a bug is encountered.

To help those scripts work across different build plans, we rely on Bamboo’s plan variables, where you define a variable name and an associated value. Those are then accessible by the scripts as environment variables.

We then extended the build plans to work on both the develop and master branches. Here, Bamboo allows you to override the value of specified variables. For example, the build plan might default to specifying that jekyll_conf_file has a value of “_config.yml,_config-staging.yml”. The master branch variant would then override that value to be “_config.yml,_config-production.yml”.

The method used to trigger the builds automatically has changed over time because we’ve changed the repository structure, GitHub has changed the service offerings and we’ve started doing more to tightly integrate Bamboo with GitHub so I’m not going to go into the details on that just yet.

Index page: Tips, tricks and notes on building Jekyll-based websites

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: