Testing Feature Branches Remotely with Grunt

At my current job we are working on multiple features simultaneously, using git feature branches. We have a Jenkins build server which we use for integration testing of the master branch, which runs about 20 jobs simultaneously for Protractor and Fitnesse tests. An individual job typically takes around 10 minutes to complete.

Our policy is to keep the master branch production ready at all times. Therefore we have a review process in place that should assure that feature branches are only pushed to master when they can’t break the application.
This all works very well as long as the feature which you are working on requires only one or two integration test suites to test its functionality. But every once in a while you’re working on something that could have effects all over the application, and you would like to run a larger number of integration test suites. And of course before you merge your feature branch to master.
Running all the integration suites on your local machine would take too way much time. And Jenkins is configured to run all its suites against the master branch. So what to do?

The Solution

In this article I’m going to show a solution that we developed for this problem, which lets us start multiple remote Jenkins jobs on the branch that we are working on. This way we can continue working on our local machine while Jenkins is running integration tests on the build server.
Most of our integration suites run against our frontend modules, and for those modules we use grunt as our build tool.
Therefore the most practical step was extend grunt with a task for starting the integration tests on Jenkins: we’d like to type ‘grunt jenkins’ and then grunt should figure out which branch we have checked out, send that information to Jenkins, and start all the integration suites.
To accomplish that we need to take the following steps:

• Have some Jenkins integration tests suites which can take a git branch as a parameter
• Create a custom grunt task called ‘jenkins’
• Let the grunt jenkins task figure out which branch we have checked out
• Let the grunt jenkins task start a bunch of remote jenkins jobs with the branch as a parameter

The Parameterized Jenkins job

Jenkins offers the feature to configure your build with a parameter. Here is how we do it:
In the configuration of a Jenkins job there’s a little checkbox saying ‘the build is parameterized’. Upon checking it, you can enter a parameter name, which will be available in your Jenkins build script.
We’ll add a parameter called BRANCH, like in the screenshot below:

Then in our Jenkins build script, we can check if the parameter is set, and if this is the case, check out the branch. It will look something like this:

[bash]
git fetch
if [[ -n “$BRANCH” ]]; then
git checkout -f $BRANCH
git pull
else
git checkout -f ${PROMOTED_GIT_COMMIT-“origin/master”}
fi
[/bash]

What’s nice about our parameterized build job is that we can invoke it via a Rest call and include our parameter as a query parameter. I’ll show that later on.

Our custom ‘jenkins’ Grunt task

In our grunt.js configuration file, we can load custom tasks. The following snippet loads all files in the conf/grunt/tasks folder.

[javascript]
grunt.loadTasks(‘conf/grunt/tasks’);
[/javascript]

In our tasks folder we create a jenkins.js file, containing our custom jenkins task.
The next thing to do is to retrieve the name of the branch which we have checked out on our machine. There’s a grunt plugin called ‘gitinfo’ which will help us with that.
When the gitinfo plugin is invoked it will add a section to the grunt configuration which contains, amongst others, the name of our current local branch:

[javascript]
module.exports = function (grunt) {
grunt.registerTask(‘jenkins’, [‘gitinfo’, ‘build-branch’]);

grunt.registerTask(‘build-branch’, function () {
var git = grunt.config().gitinfo;
grunt.log.ok(‘Building branch: ‘ + git.local.branch.current.name);
[/javascript]

And now we can start our parameterized job with the correct value for the branch parameter, like this:

[javascript]
var request = require(‘request’);

var jenkinsUser = ‘your username’;
var jenkinsPassword = ‘your password’;
var jenkinsHost = ‘your jenkins host’;
var job = ‘my-parameterized-integration-suite’;

var url = ‘https://’ + jenkinsUser + ‘:’ + jenkinsHost + ‘@’ + jenkinsHost + ‘:8080/job/’ + job + ‘/buildWithParameters?BRANCH=’ + git.local.branch.current.name + ‘&delay=0sec’;

request({
url: url,
method: ‘POST’
},
jobFinished(job));
});
[/javascript]

First we acquire a reference to the ‘request’ package. This is a simple Node package that lets you perform http requests.
We then build the Rest url; to connect to jenkins we need to supply our Jenkins username and password.
And finally we post a request to the Rest endpoint of Jenkins, which will start our job. We supply a callback called ‘jobFinished’.

Putting it all together: starting multiple jobs

With these steps in place, we have a new grunt task which we can invoke with ‘grunt jenkins’ from the commandline, and which will start a Jenkins job on the feature branch that we have checked out locally.
But this will only be useful if our grunt jenkins task is able to start not just one job, but a bunch of them.
Here is the full source code of the jenkins.js file. It has a (hardcoded) array of jobs, starts all of them and keeps track of how many of them have finished:
[javascript]
module.exports = function (grunt) {

grunt.registerTask(‘jenkins’, [‘gitinfo’, ‘build-branch’]);

grunt.registerTask(‘build-branch’, function () {
var request = require(‘request’);

var jenkinsUser = ‘your username’;
var jenkinsPassword = ‘your password’;
var jenkinsHost = ‘your jenkins host’;

var jobs = [
‘my-parameterized-integration-suite-1’,
‘my-parameterized-integration-suite-2’,
‘my-parameterized-integration-suite-3’,
‘my-parameterized-integration-suite-4’,
‘my-parameterized-integration-suite-5’
];
var git = grunt.config().gitinfo;
var done = this.async();
var jobCounter = 0;

grunt.log.writeln();
grunt.log.ok(‘Building branch: ‘ + git.local.branch.current.name);
grunt.log.writeln();

function jobFinished (job) {
return function (error, response, body) {
jobCounter++;
grunt.log.ok(‘[‘ + jobCounter + ‘/’ + jobs.length + ‘] Started: ‘ + job);

if (error) {
grunt.log.error(‘Error: ‘ + error + (response ? ‘, status: ‘ + response.statusCode : ”));
} else if (response.statusCode === 301) {
grunt.log.writeln(‘See: ‘ + response.headers.location);
}

if (body) {
grunt.log.writeln(body);
}

if (jobCounter === jobs.length) {
grunt.log.ok();
done();
}
};
}

jobs.forEach(function (job, i) {
var url = ‘https://’ + jenkinsUser + ‘:’ + jenkinsHost + ‘@’ + jenkinsHost + ‘:8080/job/’ + job + ‘/buildWithParameters?BRANCH=’ + git.local.branch.current.name + ‘&delay=0sec’;
grunt.log.ok(‘[‘ + (i + 1) + ‘/’ + jobs.length + ‘] Starting: ‘ + job);

request({
url: url,
method: ‘POST’
},
jobFinished(job));
});

grunt.log.ok();

});
};
[/javascript]

And here’s the console output:

$ grunt jenkins
Running “gitinfo” task

Running “build-branch” task

>> Building branch: my-feature-branch

>> [1/5] Starting: my-parameterized-integration-suite-1
>> [2/5] Starting: my-parameterized-integration-suite-2
>> [3/5] Starting: my-parameterized-integration-suite-3
>> [4/5] Starting: my-parameterized-integration-suite-4
>> [5/5] Starting: my-parameterized-integration-suite-5
OK
>> [1/5] Started: my-parameterized-integration-suite-1
>> [2/5] Started: my-parameterized-integration-suite-2
>> [3/5] Started: my-parameterized-integration-suite-3
>> [4/5] Started: my-parameterized-integration-suite-4
>> [5/5] Started: my-parameterized-integration-suite-5
OK

Done, without errors.

Qxperts. We empower companies to deliver reliable & high-quality software. Any questions? We are here to help! www.qxperts.io