CBD, SOA, whats next?

Most engineering disciplines have managed to split their problems into one that of components assembly. Individual components are well defined and engineered to satisfy a very concise set of requirements. The system level problem then reduces to selecting proper components and assembling them to satisfy overall system's requirement.

Doing things this way has its benefits and it is well established by other branches of engineering. Software engineering is trying to do the same, but without much success. Atleast in business IT world. We have tried component based development (CBD) and now trying SOA. But we are not any closer to the pannacea of assembly of components.

What is that constraint, which is preventing software engineering to move to next level? Implicit in this question is an assumption that software engineering wants to move to a component assembly world. Well, I am not sure that is true either. And I am not alone in this thinking, see this post.

So the problem is not entirely technical. It is also of business motivation, and viable business models. In other branches of engineering, craft became engineering, when it promised to bring prosperity to all stakeholders. In software engineering, what is that business model, which will let multitudes of component makers flourish and big guns will just concentrate on component assembly? Is it standardisation effort, that is lacking? Is ultimate end IT user really interested in such component assembly world? Because in business IT world, the enterprise IT is main differentiator for an enterprise, which prevents it from being commoditized as a service or product provider.

These are the hard questions that industry must grapple with. Otherwise, every couple of years we will have, a wave, be it CBD or SOA and as an industry we'll not move much.

Each such wave (CBD, SOA) takes us further than today. But to make that transition to next level we need an industry wide push and not vendor specific attempts. And this must include all stakeholders - viz. business IT consumers, service providers and product vendors. Perhaps OMG can take lead?

SOA: Reuse, of Interface or Capability?

SOA reusability debate is bit confused. For more clarity, we must understand the difference between service interface and capability behind it. A service exposes some interface which is backed by a capability. From a service consumer's perspective the reusability of interface is enough. Whereas from a builder's perspective the reusability of capability is more important. It is the later which is difficult (or impossible in some cases). It is possible that same interface can be mapped to multiple capabilities. For example an interface can be mapped to multiple capabilities based on Quality of Service (QoS) needs specified in interface. (Well, this is not possible right now in most implementations). The QoS needs like performance, scalability, availability, currency, data quality etc. would determine the binding of interface to capability. Sometimes even functional needs of services may force an interface to be bound to multiple capabilities developed over period of time. The problem is how does one control the proliferation of overlapping capabilities.

The reason why overlapping capabilities get developed are not straight forward. Sometimes it is weak governance, sometimes it is for QoS purposes, sometimes it is trade-off made to achieve results for a business facing project. This post by Todd Biske has elaborated about the last aspect. In practice one can address governance issues but capability duplication cannot be totally eliminated. The QoS needs will force multiple capabilities for same interface in some cases. There may be business drivers which will force a tactical trade-off to achieve larger business benefits. Well, one way to avoid trade-offs is to plan for the future, which I have suggested in this post. Still such overlapping capabilities will get built.

So what should one do? Well, one way out is to refactor and streamline the capabilities that are built. Here SOA would help, as consumers are not affected when capabilities change, as long as interface is same. So one should be refactoring and cleaning capabilities, because up-front reusable capabilties are hard to achieve in a working IT organisations. Business leaders must realize this reality and make funding available for these kinds of activities.

Enterprise IT Oracle

It is very surprising that so many competent and smart people come together in some Enetrprise IT organisation, then create a mess that is unmanageable and beyond their collective capabilities. It is not always a case of inaptitude or incompetence of individuals, but it is the nature of the beast. If we were to understand the problems of enterprise IT, we should try to analyze the root causes.

To me enterprise IT is driven by four major drivers,
1. Own business needs
2. Competative scenario
3. Regulatory compliance
4. Technology changes

These four drivers have different rate of change. Cost of not attending to those changes, is different too. Hence priority to attend to these changes is different. Enterprise IT is like an assembly line in action. And to make matter worse, it is an assembly line which can't be stopped for maintenance. So the four drivers combined with the fact that you never get any breathing space to fix enterprise IT, results in many tactical decisions which finally leads to mess of unmanageable proportions.

Ability of enterprise IT to respond to the drivers is constrained by capability of its own IT organisation, capability of its suppliers and capability of other stakeholders. Enterprise IT does not deal only with planning and design of IT but also about building, governing and sustaining too. This requires collective effort from IT organisation, suppliers and stakeholders, hence any mechanism to avoid the mess must have participation of all.

To avoid the mess EA must plan for future based on these four drivers and remember to make that plan as flexible as the rate of change within most important driver of the drivers listed above. When plan changes, it can accomodate important of the latest changes within other drivers too. The capability of IT orgnisation, supplier and stakeholders puts constraints on any plan that is created. This capability building also must be addressed within the plan itself. This planning is based on tracking of drivers and constraints. A sub-organisation within enterprise architecture community must own this. This organisation can then aptly be called Enterprise IT Oracle (An Oracle is a person or agency considered to be a source of wise counsel or prophetic opinion; an infallible authority. Not to be confused with Larry Ellison's Oracle corporation).

Web 2.0 and enterprise data management

This is a great post that every one who would like to understand web 2.0 in enterprise context must read. This article by Sam Lowe provides greater clarity on Web 2.0 and its implications for enterprise data. The interesting point that is made is more than technologies,the ideas behind web 2.0 is what is going to impact the enterprises' way of managing data. The participatory nature of Web 2.0 as applied to data management is a revelation and must form part of agenda that practitioners must shape and develop. To that effect Sam has also suggested to run a workshop. I feel its a great idea and interested people can congregate and dive more deeply into these issues.