Orchestration – A symphony of Microservices

Over the years application architecture have evolved. The evolution was driven by a need to address challenges posed by real world application requirements. We started from monolithic, client/server, n-tier and API first application architectures.

Microservices is an architectural pattern that has evolved as a response to continuous delivery & continuous integration (CD/CI), infrastructure automation, domain driven design, distributed teams, application and interaction complexity among others.

Though microservices architecture has been promoted as the cure all for all modern application development ailments and is seen as the successor for API first application development, its implementation needs far more thought and pragmatism. Development teams fall for the buzz worthiness of this architectural pattern and frequently mis-implement it in practise.

Here we won’t dwell much on what Microservices architectural pattern is. Rather we will present some common pitfalls in its implementation and provide an extension pattern (Orchestration) that is pragmatic.

What Is Microservices Architecture?

Microservices architectural style is an approach to develop a single application as a suite of services which can be independently developed and deployed. Each service runs in its own process and communicates with the external world, often using REST principles. Each service is self contained and provides a set of features that can be independently serviced.

The term “Microservice Architecture” has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services. While there is no precise definition of this architectural style, there are certain common characteristics around organisation around business capability, automated deployment, intelligence in the endpoints, and decentralised control of languages and data.

Martin Fowler

A pictorial representation comparing monolithic vs micro services architectures.

Common Implementation Pitfalls

While the canonical way (see above) of developing micro services is prescribed, frequently during implementation we see some variations. Most of the time it is due to cost considerations , development teams running short on time and to reduce interaction complexity. Let us explore some “Microservices Architectures” from real world implementations.

Single Database Microservices

Though the development team starts nice and good, they make the choice of each service talking to the same database store. Reasons given are costs, reference and relational data etc. This kind of architecture evolves when a project is kicked off (either legacy or greenfield), it starts with a single service and single database. Then the team decides we need to have a microservices based architecture and they keep adding other services (or refactoring the monolithic service into its component services). Instead of having a datastore per service the team decides to keep the same database. The rationale frequently is the cost of adding a new database for proportionally lesser data is not justifiable. Also since there is deadline pressure, the whole exercise of data migration, synchronisation and additional testing is a natural barrier to segregate the database. Over a period of time, number of services increase and the same decision matrix comes in to play at every juncture. Note : Inter service interaction has been removed to add brevity and clarity.

Coarse Grained Microservices

Instead of having granular services which is based on the Single Responsibility Principle, they create a couple of coarse grained services which bundle most of the features within it. If any new service needs to be developed, they add it to the best fit coarse grained service. So over a period of time there will be no difference between this architecture and a monolithic one. As you can see from the above diagram, when a new service C’s features need to be developed, it is natural to add it to Coarse Service A instead of creating a new service. In effect this pattern evolves starting from a monolithic service to the beginnings of a micro service and then quickly degenerates to a coarse grained service. Once a service reaches critical mass, it becomes harder and harder to refactor it in to a proper micro services architecture. Note : Inter service interaction has been removed to add brevity and clarity.

Knot Anti Pattern Microservices

This type of Microservices architecture is more insidious, in the sense that it develops overtime. It starts off as a pure form micro services and then over a period of time as more fine grained services are added, the complexity of interaction manifests as an Anti Pattern called Knot. By then it is too late as significant resources and time would have already been spent on developing it. So there will be resistance all around in fighting the inertia and fixing this. Moreover organisationally there is no single role who will be willing to change this. So they continue building it with increasing complexity and its attendant issues. When the number of micro services are few (n≤3), The service interaction complexity is not that big, hence it is easy to develop. The issues start as we keep adding more and more micro services to the application. The interaction complexity increases. Potential number of inter service connections are given by : Where n is the number of micro services. As we add more micro services the complexity of inter service interactions increases exponentially. Every new consumer facing feature would potential involve a complex interaction of services. This in turn will cause hard to trace bugs. Number of edge cases also will increase, as we need to cater to individual service failures, errors, latency etc. Since each service will embed in itself the knowledge of its interactions to other services, it becomes brittle to changes. Every deployment has to be tested for all its interactions and this negates the advantage of Microservices Architecture in the first place.

Microservices Orchestration

We have till now seen how Microservices Architecture in practise leads to sub optimal implementations. Moreover these implementation patterns occur over a period of time and have very good reasons for its evolution. So what can we do to avoid this? Is there a pattern that can alleviate most if not all of the evolutionary pitfalls? Can we establish an architecture that accepts this evolutionary implementation pressure and provides a prescription for it? The answer is yes. It is Microscervices Orchestration. We create a symphony of micro services which when conducted properly produces melodious music instead of cacophony. Orchestration prevents the rise of the Knot Anti Pattern. As the name suggests the orchestration layer orchestrates the calls to various services in order to achieve a composite feature. Each service only needs to worry about input and output of its endpoints. So development, deployment and maintenance becomes easier. Each service can then be ‘owned’ by a separate team and not worry about dependencies. The orchestration layer will need to keep track of the changes and adjust itself accordingly. From a consumer standpoint it all looks seamless and consistent. This also allows for adding of new micro services independently. The new service can then be integrated in the orchestration layer asynchronously.

Conducting The Symphony

In order to conduct orchestration in a repeatable and consistent way, we need to create Recipes for each higher order service calls. Recipe : Is an ordered configuration of underlying micro services where the sequence of inputs and outputs to and from a service are defined to achieve a higher order feature.

These recipes can then be defined and implemented in the orchestration layer. This gives us the flexibility to add/modify recipes to suit any new higher order feature requests, while isolating the underlying micro services from churn. Traditional Microservices need to perform Implicit Orchestration, whereas by performing Explicit Orchestration we break the inter service dependency, remove orchestration embedding and provide for a faster and more agile development. The diagram illustrates the difference between Implicit and Explicit Orchestration.

As one can see Explicit orchestration (using recipes) can provide a higher level of abstraction which is easier to communicate, develop and maintain.

Summary

Microservices architecture is designed for adding requirements at any time. This architecture style is a very good fit for agile development processes and for distributed teams (either geographically or functionally). It is possible to implement a minimum viable product, deliver it to the user and then extend the system over time. But we need to be aware the evolutionary patterns that emerge. By applying explicit orchestration to a Microservices Architecture we can avoid many of the common implementation pitfalls.