A data-platform is nothing more than a normal (cloud) platform with some additional functionality on top to make it specific to the requirements of the data domain. Instead of the applications that run on a “normal” platform like (web)services and front-ends it runs ELTs/ETLs and data applications. 

Social media has been blamed for locking people in a bubble, only showing them news that is in line with their beliefs. This divides society into different groups that have almost nothing in common. People read what they think they want to read, never seeing a different opinion. At the same time governments and influencers have started to call for filtering. Facebook would have to filter out lies and fake news, so we all see the truth only. The problem with the filter approach is that it will cause opinions to drift toward some bottom line truisms we can all agree on. If we start fining social media for violations, the companies will get more and more conservative, and we’ll end up in a boring world. Like having a perpetually overcast sky and an eternal drizzle. Grey goo everywhere.

This is not what we need. What we need is to be confronted with opinions that differ from what we think is right. So we (i.e. Albert Brand, Arjan Molenaar and myself) started a one-day research project at Xebia, inspired by a feature of my favorite Dutch newspaper, NRC. The feature is called Twistgesprek. The format is that two people discuss a statement during the week. Their conversation is summarized and published in the Saturday paper as a back-and-forth of messages. Quite often I start with a strong opinion about the subject being discussed, but end up with a more thorough understanding of its nuances because of the discussion. Having your convictions challenged and modified is a wonderful gift.
So, the idea was to show people ideas that directly contradict each other.
more

AWS (Amazon Web Services) offers a pretty neat NoSQL database called DynamoDB. It is fast and it can scale, what more can you wish for? The thing is, as a developer you are still responsible for designing your tables in such a way, that you actually make appropriate use of the benefits DynamoDB has to offer. If you simply apply your knowledge gained using other databases, you might end up wasting money and performance.

Many organisations have a new ambition to become a data-driven organisation. In essence, this means the organisation wants to make better business decisions based on insights provided by data [4]. Data itself is not able to advise a business for better decision-making. Therefore these organisations introduce a new capability: Data & Analytics. 
This blog elaborates on how adopting DevOps principles can enhance business value creation for the world of Data & Analytics.

Asynchronous streaming solutions, such as Rx or Reactor, rely on callbacks to perform flow control (or backpressure). Without flow control, systems could overflow and eventually crash. In this post we will look at the difference between flow control in synchronous applications and their asynchronous counterparts? [more…]

There is a lot of buzz around the practice of Progressive Delivery lately. Rightfully so, as it’s a great addition to continuous delivery. By gradually exposing new versions to a subset of users, you’re further mitigating risks. As usual with new and shiny things, many of us are eager to try it out, learn about the tools and shift to this way of working. But as is also common, there are reasons to do it, reasons to not do it, and reasons you might not succeed at it. Let me save you some trouble by elaborating on a few of them. 

Welcome to my second blog in the series of five quality patterns in Agile development that can help you to deliver the right software with great quality. In my previous blog, I’ve introduced Example Mapping as a method to get to specific examples for scenarios or rules that your user story is made up of. The output of the refinement sessions are your requirements and thus your tests. In this blog, we will take a further look at these test cases and why it is important to automate these acceptance tests. Not just from a development team perspective, but also what they can bring to your business.

In a previous blog, we described a novel way for software QA for refactored code already running in production inspired by the Scientist approach GitHub published in 2016. In this blog we explain how we implemented this QA method for testing AWS Lambda functions and our experiences with that.

Nowadays, there is an increasing interest and mentioning of socio-technical engineering, socio-technical systems. And although the words do not strike as odd on its own I personally have struggled quite a bit with the different meanings of the terms and understanding the field of socio-technical systems. So in this article, I will provide a layman’s introduction to socio-technical systems. Knowledge about socio-technical engineering can help you to understand what constraints might prevent or help you to succeed in your current project.

When using AWS Lambda functions you typically want to return a response to the client ASAP. However, imagine a situation where you calculate the response for the client and want to do some actions after sending the response to the client (e.g., write some metrics). Since standard AWS Lambda functions do not allow you to execute any actions after returning the response,  the client will experience extra latency due to the other actions which must be completed first. This blog explains how to use AWS Lambda custom runtimes to reduce the added latency and still do the additional processing.