How to write an Amazon RDS service broker for Cloud Foundry

Cloud Foundry is a wonderful on-premise PaaS that makes it very easy to build, deploy while providing scalability and high availability to your stateless applications. But Cloud Foundry is really a Application Platform Service and does not provide high availability and scalability for your data. Fortunately, there is Amazon RDS, which excels in providing this as a service.
In this blog I will show you how easy it is to build, install and use a Cloud Foundry Service Broker for Amazon RDS. The broker was developed in Node.JS using the Restify framework and can be deployed as a normal Cloud Foundry application. Finally, I will point you to a skeleton service broker which you can use as the basis for your own.

Cloud Foundry Service Broker Domain

Before I race of into the details of the implementation, I would like to introduce you into the Cloud Foundry lingo. If you are aware of the lingo, just skip to the paragraph ‘AWS RDS Service Broker operations’.
Service – an external resource that can be used by an application. It can be a database, a messaging system or an external application. Commonly provided services are mysql, postgres, redis and memcached.
Service Plan – a plan specify the quality of the service and governs the amount memory, disk space, nodes etc. provided with the service.
Service Catalog – a document containing all services and service plans of a service broker.
Service Broker – a program that is capable of creating services and providing the necessary information to applications to connect to the service.
Now a service broker can provide the following operations:
Describe Services – Show me all the services this broker can provide.
Create Service – Creating an instance of a service matching a specified plan. When the service is a database, it depends on the broker what this means: It may create an entire database server, or just a new database instance, or even just a database schema. Cloud Foundry calls this ‘provisioning a service instance’.
Binding a Service – providing a specific application with the necessary information to connect to an existing service. When the service is a database, it provides the hostname, portname, database name, username and password. Depending on the service broker, the broker may even create specific credentials for each bind request/application. The Cloud Controller will store the returned credentials in a JSON document stored as an UNIX environment variable (VCAP_SERVICES).
Unbind service – depending on the service broker, undo what what done on the bind.
Destroy Service – Easy, just deleting what was created. Cloud Foundry calls this ‘deprovisioning a service instance’.
In the next paragraph I will map these operations to Amazon AWS RDS services.

AWS RDS Service Broker operations

Any Service Broker has to implement a REST API of the Cloud Foundry specification. To create the Amazon AWS RDS service broker, I had to implement four out of five methods:

describe services – returns available services and service plans
create service – call the createDBInstance operation and store generated credentials as tags in with the instance.
bind service – call the describeDBInstances operation and return the stored credentials.
delete service – just call the deleteDBInstance operation.

I implemented these REST calls using the Restify framework and the Amazon AWS RDS API for Javascript. the skeleton looks like this:
[javascript]
// get catalog
server.get(‘/v2/catalog’, function(request, response, next) {
response.send(config.catalog);
next();
});
// create service
server.put(‘/v2/service_instances/:id’, function(request, response, next) {
response.send(501, { ‘description’ : ‘create/provision service not implemented’ });
next();
});
// delete service
server.del(‘/v2/service_instances/:id’, function(req, response, next) {
response.send(501, { ‘description’ : ‘delete/unprovision service not implemented’ });
next();
});
// bind service
server.put(‘/v2/service_instances/:instance_id/service_bindings/:id’, function(req, response, next) {
response.send(501, { ‘description’ : ‘bind service not implemented’ });
next();
});
// unbind service
server.del(‘/v2/service_instances/:instance_id/service_bindings/:id’, function(req, response, next) {
response.send(501, { ‘description’ : ‘unbind service not implemented’ });
next();
});
[/javascript]
For the actual implementation of each operations on AWS RDS, I would like to refer you to the source code of aws-rds-service-broker.js on github.com .

Design decisions

That does not look all too difficult does it? Here are some of my design decisions:

Where do I store the credentials?

I store the credentials as tags on the instance. I wanted to create service broker that was completely stateless so that I could deploy it in Cloud Foundry itself. I did not want to be dependent on a complete database for a little bit of information. The tags seemed to fit the purpose.

Why does bind return the same credentials for every bind?

I wanted the bind service to be as simple as possible. I did not want to generate new user accounts and passwords, because if I did, I had even more state to maintain. Even more, I found that if I bind two applications to the same MySQL service, they could see each others data. So why bother creating users for binds? Finally, making the bind service simple, kept the unbind service even simpler because there is nothing to undo.

How to implement different service plans?

The createDBInstance operation of AWS RDS API operation, takes a JSON object as input parameter that is basically the equivalent of a plan. I just had to add an appropriate JSON record to the configuration file for each plan. See the description of the params parameter of the createDBInstance operation.

How do I create a AWS RDS service within 60 seconds?

Well, I don’t. The service broker API states that you have to create a service within the timeout of the cloud controller (which is 60 seconds), but RDS takes a whee bit more time. So the create request is initiated within seconds, but before you can bind an application to it may take a few minutes. Nothing I can do about that.

Why store the service broker credentials in environment variables?

I want the service broker to be configured upon deployment time. When the credentials are in the config file, you need to change the files of the application on each deployment.

Installation

In these instructions, I presume you have access to an AWS account and you have an installation of Cloud Foundry. I used Stackato which is a Cloud Foundy implementation by ActiveState. These instructions assume you are too!

Create a AWS IAM user
First create a AWS IAM user (cf-aws-service-broker) with at least the folllowing privileges
Assign privileges to execute AWS RDS operations
The newly created IAM user needs the privileges to create RDS databases. I used the following permissions:
[javascript]
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“rds:AddTagsToResource”,
“rds:CreateDBInstance”,
“rds:DeleteDBInstance”,
“rds:DescribeDBInstances”,
“rds:ListTagsForResource”
],
“Resource”: [
“*”
]
},
{
“Effect”: “Allow”,
“Action”: [
“iam:GetUser”
],
“Resource”: [
“*”
]
}
]
}
[/javascript]
Generate AWS access key and secret for the user ‘cf-aws-service-broker’
Create a Database Subnet
Create a database subnet ‘stackato-db-subnet-group’ in the AWS Region where you want to have the databases to be created.
Check out the service broker
[bash]
git clone https://github.com/mvanholsteijn/aws-rds-service-broker
cd aws-rds-service-broker
[/bash]
Add all your parameters as environment variables to the manifest.yml
[javascript]
applications:
– name: aws-rds-service-broker
mem: 256M
disk: 1024M
instances: 1
env:
AWS_ACCESS_KEY_ID: <fillin>
AWS_SECRET_ACCESS_KEY: <fillin>
AWS_REGION: <of db subnet group,eg eu-west-1>
AWS_DB_SUBNET_GROUP: stackato-db-subnet-group
SERVICE_BROKER_USERNAME: <fillin>
SERVICE_BROKER_PASSWORD: <fillin>
stackato:
ignores:
– .git
– bin
– node_modules
[/javascript]
Deploy the service broker
[bash]
stackato target <your-service-broker> –skip-ssl-validation
stackato login
push
[/bash]
Install the service broker
This script is a cunning implementation which create the service broker in Cloud Foundry and makes all the plans publicly available. In stackato we use the curl commands to achieve this. This script requires you to have installed jq, the wonderful JSON command line processor by Stephen Dolan.
[bash]
bin/install-service-broker.sh
[/bash]

Now you can use the service broker!

Using the Service Broker

Now we are ready to use the service broker.

Deploy a sample application
[bash]
$ git clone https://github.com/mvanholsteijn/paas-monitor
$ stackato push -n
[/bash]
Create a service for the mysql services
[bash]
$ stackato create-service
1. filesystem 1.0, by core
2. mysql
3. mysql 5.5, by core
4. postgres
5. postgresql 9.1, by core
6. redis 2.8, by core
7. user-provided
Which kind to provision:? 2
1. 10gb: 10Gb HA MySQL database.
2. default: Small 5Gb non-HA MySQL database
Please select the service plan to enact:? 2
Creating new service [mysql-844b1] … OK
[/bash]
Bind the service to the application
[bash]
stackato bind-service mysql-844b1 paas-monitor
Binding mysql-844b1 to paas-monitor … Error 10001: Service broker error: No endpoint set on the instance ‘cfdb-3529e5764’. The instance is in state ‘creating’. please retry a few minutes later (500)
[/bash]
retry until the database is actually created (3-10 minutes on AWS)
[bash]
stackato bind-service mysql-844b1 paas-monitor
Binding mysql-844d1 to paas-monitor …
Stopping Application [paas-monitor] … OK
Starting Application [paas-monitor] …
OK
https://paas-monitor.<your-api-endpoint>/ deployed
[/bash]
Check the environment of the application
[bash]
curl -s https://paas-monitor.<your-api-endpoint>/environment | jq .DATABASE_URL
“mysql://root:e1zfMf7OXeq3@cfdb-3529e5764.c1ktcm2kjsfu.eu-central-1.rds.amazonaws.com:3306/mydb”
[/bash]
As you can see the credentials for the newly created database has been inserted into the environment of the application.

Creating your own service broker

If you want to create your own service broker in Node.JS you may find the Skeleton Service Broker a good starting point. It includes a number of utilities to test your broker in the bin directory.

catalog.sh – calls the catalog operation
provision.sh – calls the create operation
unprovision.sh – call the delete operation
bind.sh – calls the bind operation on a specified instance
unbind.sh – calls the unbind operation on a specified instance and bind id.
list.sh – calls the list all service instances operation
getenv.sh – gets the environment variables of an CF applications as sourceable output
install-service-broker.sh – installs the application and makes all plans public.
docurl.sh – calls the stackato CURL operation.

getenv.sh, install-service-broker.sh and provision.sh require jq to be installed.

Conclusion

As you can see, it is quite easy to create your own Cloud Foundry service broker!