Scripting Deployit, Part 2

Like all software, Deployit plugins should be build automatically, deployed and tested whenever possible. Below I will explain how to setup continuous integration and testing for Deployit plugins. The general idea is that if you’re creating a larger set of plugins for Deployit, it makes sense to try to build and deploy them as often as possible so you can catch errors early. We found that it is really easy to create plugins that work fine in isolation but fail when they are deployed together. Luckily it is also easy to catch these kinds of errors as I will show below.

This work is part of a larger effort to describe and explain Deployit. I’ve written a previous blog you can find here: https://xebia.com/blog/2013/03/24/scripting-deployit/.
Code for the examples and plugins can be found on GitHub:
git clone git://github.com/jvermeir/DeployitBlogs.git
cd DeployitBlogs
git checkout ec6b748.

I’ve borrowed heavily from work I did with my colleague Joris de Winne.
I will use a couple of plugins that define a new host type. They serve only to show how the build and test process works so please ignore their contents. In the source tree you can find them in the ‘citest/test’ directory. Each plugin comes with a buildAndCopy.sh script that does no more than copy a jar file to the plugins directory so it can be picked up by Deployit.
To run the examples you’ll need a shell and change to the directory named ‘citest’ in ‘DeployitBlogs’, the root of a project on GitHub I’ve described in a previous post.
The build and deploy script is started by typing
./basicDeployitStartupTest.sh p1

which will do a clean install of Deployit, copy the plugin named p1 and finally start Deployit to find out if it still works. The script will log messages about the steps it takes, i.e.:
findAndTerminateDeployitProces
installDeployit
copyPluginsFromDistribution
buildAndCopyPlugins
validate

The first step is to find and terminate a Deployit process that may already be running on the test server. This is done by looking for a Java process that runs Deployits main class com.xebialabs.deployit.DeployitBootstrapper, and then terminating this process. This may fail of course, meaning you’ll get errors later on.
Next, installDeployit() removes a previous version and unzips the Deployit server archive. The archive I’ve used is not the standard distribution but a version that was created from an existing install. Deployit needs a couple of answers when it starts for the first time, so all I did was run the installer, answer the questions, stop Deployit and zip the full directory into a new installer file that is unzipped by the installDeployit() method. So for this step to work you’ll have to do the same and store the results in $ROOTDIR/resources/deployit-$DEPLOYIT_VERSION-server-setup.zip. Refer to the script to find out where the shell variables should point to.
The third step, copyPluginsFromDistribution(), copies all default plugins from the available-plugins directory to the plugins directory so we can use all of Deployits features. You can change this if you don’t need all plugins for your use case. The third line in this method uses Linux find to copy even more plugins that are neither part of the standard distribution nor of your project. An example could be the weblogic plugin. These plugins should be copied in the lib directory under citest.
The fourth step, buildAndCopyPlugins(), builds a list of plugins you want to test and copies them to the plugins directory. The function takes a list of plugin names separated by a blank in a variable called $PLUGIN_NAMES. It then loops over this list and calls a script named buildAndCopy.sh that should reside in the root directory of each plugin. The sample plugins in test/success show how this works. In the real world you might start a Jenkins build, copy an artifact from a repository or call Maven.
The only requirement is that a build should log to the console and print a line with the text ‘BUILD FAILURE’ if something goes wrong.
Finally validate() tries to start Deployit and waits for the message “You can now point your browser to” to appear in the console log. The method checks every 5 seconds for a minute and reports the result.
To experiment you can use the two sets of plugins provided in the test/success and test/failure directories.
./basicDeployitStartupTest.sh p1
deploys a single plugin named p1 from test/success. This test should report success.
./basicDeployitStartupTest.sh -p test/failure p1 p2
should report failure because it tries to deploy the same plugin twice. This takes a while because the test keeps looking for a success message but an error is logged in stead. There’s some room for improvement here, I guess.
We’ve successfully used a predecessor of this script to keep a grip on a project with a large set of plugins defining many custom types. We ran the script as a Jenkins job triggered by a commit on Subversion. In our setting it proved a useful tool to catch type definition errors early on.