Design Rules: the Power of Modularity

I recently read a book called ‘Design Rules, Vol. 1: The Power of Modularity’ written by Carliss Y. Baldwin and Kim B. Clark. It was pointed out to me by Adrian Colyer during the excellent AOP training mentioned in the previous post.
The title ‘Design Rules’, note the plural ‘Rules’, is something of a misnomer. In fact the authors describe only a single rule, albeit a very important one. They argue that the computer industry started its spectacular growth after clever engineers at IBM decided to break up the design of the 360 family into a set of modules that depended on each other through interfaces.

The designers of such a module could do more or less as the pleased as long as they implemented the interface correctly. This allowed different implementations of e.g. the disk subsystem, fostering competition and better designs of subsystems. As long as an implementation respects the interface the implementation can be anything, including revolutionary new solutions. The price to pay is that changing an interface is costly: changing the bus standard on a PC e.g. would make a lot of hardware obsolete and therefore requires lots of engineering effort. Rigidly defined interfaces introduce stability that may degenerate into rigidity.
The authors don’t describe a method to create a good modular design, but they offer a check that allows you to find out whether you’re going in the right direction. The concept is that of a dependency matrix. Every feature of a design is listed in a column as well as in a row at the top of the matrix. Then they place an X at the intersection of a row and column if the feature in the row depends on the feature at the top of the column. A design can be modularized if it is possible to group a set of features that depend on each other but not on features outside of the group. This quality is easy to spot in a matrix form, much easier than I can describe it in words.
The book describes two examples extensively: the development of the IBM 360 family and the concepts of the Unix implementation on a PDP-11. I thoroughly enjoyed reading these descriptions of early computer history. True, the authors could have gotten their point across in 100 pages in stead of 483, but if you view the book as a description of computer history it really works out fine.
The authors imply that once a system is modularized you may calculate how many different implementations of a module can be tried in an economically sensible way. This is where they lose me in formulas that remain unproven, but make intuitive sense. It is easy to see that as long as an implementation satisfies the rules of the interface it can be replaced by another implementation. This means you can try and find a better version of, say, a network interface card as long as it fits in a PCI slot and understands the signals on the bus.
I believe the modularization trick should work for software as well. The book focuses on hardware, where the benefits of modularization are more apparent, but it is easy to see that we need to split up large problems into smaller ones that can be solved and evolve independently. I did exactly this at a customer in 2005. The system they built had become very rigid because every change would bounce around all of the system in unpredictable ways. Customer features depended on product features and vice versa turning changes into a real leap of faith. We first drew a module diagram, i.e. a single-page lines-and-boxed diagram describing the dependencies between the main components. Circular dependencies were not allowed. Then we started to refactor the code to satisfy the desired dependencies. The effort took four developers less than six months to complete and resulted in a system with modules that could be changed independently.
I can hear some of you whisper ‘XML’ in the background, but we didn’t go that way. All we needed was Java interfaces: code in a module depends on the interfaces in other modules, not on implementations. If we wanted to we may substitute direct Java calls by EJBs or SOAP services, but that was not necessary so we didn’t.
The software equivalent of the PC bus is not EJB and it isn’t SOAP either. The software equivalent of the PC bus is a plain old Java Interface.