Developing a SOA-based integration layer framework: goals

A few years ago I was asked by one of our customers to help them make better use of their integration layer. Ever since then me and my team have been working on a framework in support of that. This is the second in a series of blogs on the development of our framework, and discusses the goals we had to reach.
In the previous blog of this series I mentioned the business needs that we had to address:

• Efficiency in business processes
• Consistency in data representation
• Flexibility and time-to-market accelerated by the IT department.

Based on these the goals described below were set.

Goals

1. The framework should make it possible to connect systems from different business domains as quickly as possible.
Quick being relative here of course: in the old version of the integration layer it could take up to 40 hours of work (that’s total time spend) to provide a simple Web service proxy. With developers being impatient as they are, this meant that sometimes the integration layer was completely passed by, making it hard to get a grasp on the different out-of-domain consumers of a service, as well as leading to all kinds of security risks.
2.      The framework should make it possible for a consumer to always invoke services in other domains in the same way.
As there were no real rules for Web services design, the service landscape was ‘disparate’ to say the least. Even worse; on several places standard protocols like SOAP were not used. Also, there was no service registry present, so even finding a particular service, and then discovering how to invoke it, could be time-consuming. Our goal was to provide a federated endpoint layer for all consumers of out-of-domain services.
3.      The framework should be based on SOA principles.
Long before I joined up the customer made the decision to use SOA. Fancy reports with lots of big words were written and then put into a drawer. Being an SOA advocate myself, I took it upon me to follow the 8 service-orientation design principles to a meaningful extent. Not because it looks so good on a PowerPoint-slide, but because principles like service loose coupling and standardized service contract actually make a lot of sense when building an integration layer framework.
Related to this goal is also the fact that we use industry standards as much as possible, and only divert from them when absolutely necessary. That said, we did pull a few tricks out of our sleeve on which I’ll come back in a future blog.
4.      The framework should provide generic features.
The funny thing about an integration layer is that it’s centralized, and only delivers added value when the features it provides are meaningful to all consumers, not just the project asking for them. Heck, even the project we booked our hours on was called ‘generic features integration layer’, so that tells you how important this goal is.
5.      The framework should consist of recomposable components.
This one is a direct result of goal #1: in order to make it possible to provide quick connections between different systems, it was necessary to build several generic framework services which could be composed as part of specific solutions.
6.      The framework should be able to run on different platforms.
Now this is an important one: basically I’m discussing the concept of the framework in this blog series, and not so much the implementation (although there were quite strict rules about that as well, i.e.: it should use this expensive piece of hardware the company had bought). This meant that in order for this goal to be reached, we had to be very careful about finding the balance between what the integration platform could do, and what we actually wanted.
7.      The framework should be built in an agile way.
With both me and a fellow Xebia consultant (both avid believers in the Agile methodology) having just kickstarted the first successful Scrum project at our customer, and being part of the first team responsible for the new integration layer, it was no wonder that this goal was set as well. More important though: it’s a perfect way of tackling the issues that are really important first, and gradually adding new features to the framework.

Conclusion

I think that in the end we’ve managed to get a near-perfect score, although there’s always room for improvement. Also, the first of the business needs is not addressed at all: we are thinking about doing that in the next version of the framework.
That’s it; next time I’ll address the challenges that we had to meet.