Innovation day at KLM: Persistence with Docker containers

On 3th of July KLM and Cargonauts joined forces at KLM headquarters for an innovation day. The goal was to share knowledge and find out how to properly do “Persistence with Docker containers”.
Persistence is data that you want to have available after the reboot, and to make it more complex in some cases you also want to share that data over multiple nodes. Examples of this are an upload folder that is shared or a database. Our innovation day case is focusing on a MySQL database, we want to find out how we can host MySQL data reliable and highly available.

Persistence with Docker containers poses the same dilemmas as when you don’t use them but with more options to chose from. Eventually those options could be summarized to this:

1. Don’t solve the problem on the Docker platform; Solve it at the application level by using a distributed database like Cassandra.
2. Don’t solve the problem on the Docker platform; Use a SAAS or some technology that provides a high available storage to your host.
3. Do fix it on the Docker platform, so you are not tied to a specific solution. This will allows you to deploy everywhere as long as Docker is available.

Since this was an innovation day and we are container enthusiasts, we focused on the last option. To make it more explicit we decided to try and investigate into these two possible solutions for our problem:

1. Provide the host with a highly available distributed file system with GlusterFS. This will allow us to start a container anywhere and move it freely because the data is available anywhere on the platform.
2. GlusterFS might not provide the best performance due to its distributed nature. So to have better performance we need to have the data available on the host. For this we investigated Flocker.

Note: We realise that these two solutions only solve part of the problem because what happens if the data gets corrupted? To solve that we still need some kind of a snapshot/backup solution.
Having decided on the approach we split the group into two teams, team 1 focused on GlusterFS and team 2 focused on Flocker. We will post their stories in the coming days. Stay tuned!
The innovation day went exactly as an innovation day should go, with a lot of enthusiasm followed with some disappointments which led to small victories and unanswered questions, but with great new knowledge and a clear vision where to focus your energy next.
We would like to thank KLM for hosting the innovation day!