Rajith’s Column

My friend Matt brought to my attention an article on why MapReduce is a step backward, written by David J. DeWitt and Michael Stonebraker. I also stumbled on this blog entry written by Mark CC from Google. It is sad to see that most people advocate a particular solution to every problem imaginable and then refuse to look at anything else from a neutral pov. I let you draw your own conclusions.

It’s been almost 3 months since I attended the Colorado Software Summit (2007), but due to studies and the Red Hat Messaging launch I never got around to blog about the notes I took during Dan’s talk on “Architecting for latency”. I managed to blog about his other talk on Scalability here. Well better late than never. So here it is.

One of the key points he mentioned was that people often ignored latency and tried to work around it instead of embracing the reality. Which is exactly the second fallacy of distributed computing - Latency is zero. Understanding this reality is important especially if you are involved in systems that are distributed geographically. Here are some of the tips I noted down.

• Try to serve users from where they are located.
  Move latency from users into your network. This will move the complexity into your system, but it will improve the user perceived performance
  Ex: If you serve your European customers from a data center in US, your customers will experience some delays. If you add a data center in Europe then it will improve the customer experience, but now it will add more complexity into your system as you would need to deal with consistency between the data centers ..etc
• Co-locating your services is a good strategy, but it may not be possible all the time. So always think about how you would architect your system or components so there are no issues if you move them apart from 10ms to 100ms.
• Don’t couple your system to the hardware or network topology. Upgrading might be a nightmare
• Trying to achieve global data consistency limits options
  Think about what your business can tolerate when it comes to inconsistent state? Trying to achieve global consistency can make things very complicated.
  Keep your users in one data center. Bouncing them between data centers will force your to maintain global consistency. Tell them that the data is valid within x secs.
• Prioritize according to user needs
  Here is a nice example given by Dan. SLA for seeing the payments due on an invoice is 5 mins and the customer never complains:)
  SLA for seeing money in a sellers account is 10 secs, or else the users get a bit upset.
• If you are recovering from a failure, reduce serviceability until your system gets back to normal state. This will prevent overloading the system and increasing latency.
• Use distributed transactions carefully
  Here is an example.
  • Component C in the first configuration is complicated as it needs to ensure both data bases are updated. DB2 can be in a different geographical location and updating it can add a lot to latency.
  • In configuration two, C is smaller and simple. We can introduce other components to process the data without impacting the user perceived performance for the client.
  • Another advantage is that C can do a transaction even when DB2 is down.

Mark Baker has hung up his boots on the REST vs SOAP debate. I appreciate his effort in building awareness about the value of REST and convincing people that it provides a solid basis for designing distributed systems. In the same post, Mark also says that, “The war really has been won”. Other REST folks including Stefan Tilkov says more or less the same thing too. I don’t know. In my humble opinion, a few years from now when the systems we design/build today using a RESTful, WS-* (or whatever) approach will show us which approach yields the better result in terms of scalability, extensibility, reliability, interoperability, flexibility, versioning, reusability ..etc. Sure the REST folks would say the web has being there for a while now and it works. But usually there is a human on the other side that drives the interaction. So it remains to be seen whether it will be the same with application-to-application interactions as well.
After all the hype surrounding REST,WS-* etc.. dies out and when people have enough experience building real world applications using both approaches and realizes the advantages and disadvantages of each approach there will be less debate as to which is better, or whether we need both approaches ..etc. The answers to these questions will become more clear to the ordinary folks in time. I for one will have an open mind and is very interested to see the outcome of all of this.

The REST vs SOAP debate has been more emotional/religious and less technical as of late. Several folks burned bridges due to insidious remarks, inflammatory comments and even personal attacks. Irrespective of the technical merits one should be able to tolerate/appreciate differences of opinions and debate in a more disciplined manner than resorting to personal attacks or inflammatory comments. Even if you are the most intelligent person in the world it doesn’t matter if you can’t put forward your technical arguments without degrading your own self by making inflammatory comments or personal attacks.