SOA learnings from Amazon and Facebook

I came across this interesting and very frank article written by Steve Yegge. You can read is here.

Having worked at Amazon for close to 4 years, I could not but agree with Steve's take on Amazon, Platform, SOA and 'Eat Your Own Dog food' concept. Thinking and designing in terms of services comes naturally to SDEs at Amazon.

It is interesting to understand what platform means. Platform is a term for technology that enables the creation of products and processes that support present or future development.

If we talk about software platform, it is a software program that makes services available to other software programs through Application Programming Interfaces (APIs).  e.g. the Java platform is a generic platform that runs programs written in the Java language.

Two notable characteristics about software platforms that I want to quote from the book "Invisible Engines":

Software platforms are at the heart of “ecosystems” that consist of mutually dependent communities of businesses and consumers that have a symbiotic relationship with the platform. 

"Software platform businesses have at least two sides. Software platforms consist of services that are often made available to developers through APIs. They are also made available to computer users, but those computer users typically avail themselves of API-based services by buying applications that in turn use APIs. It is only a slight exaggeration to say that all software platform makers all the time invest in getting both developers and users to use their platforms."

Look at the platform offerings from Amazon via Amazon Web Services(AWS) and Facebook. They provides developers with:

• Software services available through APIs

• Software development kits that facilitate writing applications

• A “developer zone” that provides, for example, access to tools, sample code, and technical support

• Member forums for online discussions with other developers

The AWS offerings now allows creating and hosting enterprise applications completely on the cloud.

Thinking in terms of platform and API allows great deal of innovation. In recent times with the advent of free web services, numerous mashups were created, which in turn created value for end users. Availability of Android OS and the 'apps' it can run has created a win-win situation for developers, hardware manufacturers and end-users.

A good indicator for creating a platform is when a big pieces of inter-dependent business functionalities needs to be realized and the future requirements are unknown or vague at best.

While deciding about creating a platform, it is important to have a clear vision. Some basic questions that needs to be answered are:

• What problem in the platform solving?
• Is the platform reusable? Can many clients benefit from it?
• Who will use it and how?

And then more specific questions like what APIs should be provided and many more. Infact, creating and externalizing a software platform is similar to writing a business plan and then launching a startup! It is learning all the way.

Often while interacting with other software engineers/architects, I find it very difficult to explain the need to go with SOA or think in terms of platform . Software engineers are inherently opposed to the idea. Mainly because they have not seen and experienced the architecture before or 'fear' the additional complexity that comes with it. The most common excuse for not using web services I hear is that SOAP is heavy weight, that web services add complexity and even phrases like "too many moving parts" and "we don't need SOA now"!

Indeed, these are valid statements but the benefits of SOA far exceed these short comings. Companies like Amazon has clearly demonstrated it. Not only does SOA solve business problems, it lowers costs and create values for end customers. With low margins in E-Retailing, Amazon had to adopt it.

The biggest hurdle software developers have to cross is the ability to 'think' in terms of platform and to realize it.

More interesting articles on SOA:
Software design at it's best
SOA @ Amazon

Software design at it's best!

Once upon a time, in one of the software projects, team A was collaborating with team B to develop a workflow system. The main workflow would invoke other smaller workflow - an orchestration system was the need of the day. A meeting was held and the agenda was to integrate the two workflows.

Day 1:
Team B: We have a web service and the only API 'pushMessage' should be called from your code.
Team A: Why remoting? We need a jar.
Team B: That is a very good question. The workflow is a system and not a library. We want to avoid code inter dependency. Code deployment and maintainability is easier. Service is reusable, and in this case, it encapsulates a complete workflow.
Team A: Generating web service client is pain ...(sorry but I don't remember the following words as I was rotfl; in my mind of course). We need a jar.
Team B: Generating web service client is trivial. It  takes about 15 mins. We can do it for you.
Team A: Ok, let us discuss this again tomorrow.

Day 2:
Team A: We have a great idea. Let us merge the code of these 2 separate projects. We will combine all the APIs and create 1 WSDL. It will run very fast.
Team B: (After recovering from the shock) These are 2 distinct functional components. The APIs have no relation to each other. There is a clear boundary between the 2 sub systems.
Team A: It is complex.
Team B: What is complex?
Team A: Making a service call from our code is complex.
Team B: Making a service call is complex? How?
Team A: Yes, we have to make this service call inside a transaction and it is complex.
Team B: What!! When did you decide to make a service call inside a database transaction?
Team A: Today. And we have a problem. It is complex.
Team B: What is complex?
Team A:  Making a service call from our code is complex.
Team B: Write 0 lines of code and you get code that has zero complexity .
Team A: Does writing 0 lines of code makes the code fast?
Team B: Yes, the code runs at infinite speed!
Team A: And can I apply encapsulation to the design?
Team B: All team members faint.

You make also like this humorous post on NoSQL vs RDBMS.