Musings on Tech

I keep six honest serving-men (They taught me all I knew); Their names are What and Why and When And How and Where and Who. I send them over land and sea, I send them east and west; But after they have worked for me, I give them all a rest. I let them rest from nine till five, For I am busy then, As well as breakfast, lunch, and tea, For they are hungry men. But different folk have different views;   I know a person small- She keeps ten million serving-men, Who get no rest at all! She sends'em abroad on her own affairs, From the second she opens her eyes- One million Hows, two million Wheres, And seven million Whys!   --  Rudyard Kipling

Over the last few days, few of my friends have been asking me about NoSQL and its relevance.  I have been casually trying to answer this, but have always felt that this subject need to elaborated.  Through the slide deck, I have tried to provide a perspective on this subject. RDBMS to NoSQL. An overview. View more PowerPoint from Girish Raghavan Hope it helps. P.S:  Some of the slides ended up looking very busy. My apologies for that. This is a result of trying to balance between content (without sufficient audio explanations) and managing the size of the deck.

By Peter Deutsch   and James Gosling Essentially everyone, when they first build a distributed application, makes the following eight assumptions. All prove to be false in the long run and all cause big trouble and painful learning experiences. 1. The network is reliable 2. Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn't change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous There is a great article by Arnon Rotem-Gal-O z  explaining the same.  Read it if you are interested.  (Ref:  http://nighthacks.com/roller/jag/resource/Fallacies.html )

Definition: In theoretical computer science, the CAP Theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: Consistency (C) Availability (A) Partition Tolerance (P) According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all the three. ( Reference: Wikipedia ) Relevance and Importance: It has been over twelve years since, Eric Brewer , then a scientist at University of California Berkeley, made the conjuncture which led to what we now universally acknowledge as CAP Theorem.  But over these years, CAP theorem has changed the rules and proved to be one of the significant seeds on determining how a highly scalable and distributed computing platform can be built.  Over these twelve years, this theorem has ended up as one of the primary read for anyone who is involve...

In my previous post on this subject, I talked about few of the common problems that plague traditional data warehousing initiatives. Few of my friends after reading through the article asked me whether I am questioning the relevance of    traditional  data warehousing .    The answer to that question is  a resounding  "No".   Data warehousing do offers significant  benefits to the business, but my belief is that the practioners of the data warehousing infrastructure need to evolve beyond the traditional  models and build platforms that leverages the benefits of the traditional warehouse while complementing it with system(s) that provide quick, agile and   adaptive solutions to meet the need of the business. In this post, I am making a proposal for one such system and I am calling it an " Analytics Platform ". What is an Analytics Platform? I would define Analytics Platform as an engine which is built with a underly...

Dilbert on Agile and Extreme Programming -  Picked up from dilbert.com - Scott Adams.

Over the years, have been involved with few of the data warehousing efforts.   As a concept, I believe that having a functional and active data  ware house is essential for an organization. Data warehouses facilitate easy analysis and help analysts in gathering insights about the business.   But my practical experiences suggest that the reality is far from the expectations. Many of the data warehousing initiatives end up as a high  cost, long gestation projects with questionable end results.   I have spoken to few of my associates who are involved in the area and it appears that  quite a few of them share my view. When I query the users and intended users of the data warehouses, I hear issues like: The system is inflexible and is not able to quickly adapt to changing business needs.  By the time, the changes get implemented on the system, the analytical need for which the changes were introduced is no longer relevant. The implementors of...

Of late, have been looking into the Big Data space and Hadoop in particular.  When I started looking into it, found that there are so many products and tools related to Haddop.   Using this post summarize my discovery about Hadoop Ecosystem. Hadoop Ecosystem A small overview on each is listed below: Data Collection  - Primary objective of these is to move data into a Hadoop cluster Flume : - Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store. Developed by cloudera and currently being incubated at Apache software foundaton. The details about the same can be found here . Scribe : Scribe is a server for aggregating streaming log data. It is designed to scale to a very large number of nodes and be robust to network and node failures. Dveloped by Facebook and can be found here .  Chuckwa : Chuk...

In the last post I talked about Gamification and different concepts behind Gamification.   In this post I intend to cover why Gamification has become important. Let us accept it.  Today we live in a world which is increasingly becoming online.  More and more businesses  are moving online and one of the prime focus for these businesses is to sustain and improve the engagement levels of its users.   Gamification acts as key driver in increasing the engagement level. Before we delve into how Gamification impacts engagement; let us try to understand what "Engagement" means.  The dictionary definition of Engagement which would applicable to this scenario will be " emotional involvement or commitment". Given this definition - the first aspect to analyze is what drives this involvement and commitment.  The generally accepted hypothesis is that it is "Motivation" that drives commitment which implies that Motivation is the key contributing factor ...

Late last year  Deloitte released their report predicting top 10 emerging and disruptive technologies that is expected to play a crucial role in business in 2012.  Gamification was one of them. What is Gamification? Gamification is used as an umbrella term that covers game design thinking, concepts and techniques being applied to non gaming applications to make them more fun and engaging.  The basic goal of Gamification is to use gaming techniques to increase engagement of the users with the application. Gamification is not a new concept.  It has been there for a long time.   Airlines have been using Gamification techniques to attract and retain their user base through frequent flyer programs.  Many retail chains have been using Gamification concepts in their loyalty programs, etc.  But with the explosion of online retail, social media and social marketing, Gamification is becoming a strong tool to attract new customers, retain th...

From a collection of writings I wrote in 2007 when I first explored this methodology . To sum it up in one short sentence - " Scrum was abandoned ". Many might spin it the way they like, but the hard fact remains. This was really unfortunate given that the process during the initial days were adopted quite enthusiastically by everyone and many really tried to follow it to the best of their abilities. Now at the end of the release (which proved to be one of the more stress full releases), the whole SCRUM methodology - as adopted and followed by us is in shambles. It attracts no credibility among the team and everyone including yours truly is skeptical on the way ahead. At a personal level, I am very disappointed as I believed and continue to believe that Scrum offers significant advantages during the Software Development lifecycle. What attracted me and in which I still see merit are: a) The focus on doing the most important thing (biggest bang for the buck) first ...

From a collection of writings I wrote in 2007 when I first explored this methodology . We are almost at the middle of our release and have been trying to learn, adapt and incorporate Scrum methodology for about eight weeks now. I think this would be a right time to jot down few points on how we are progressing as I believe that, the initial euphoria/dejection about incorporating a new process has now died down and people have dirtied their hands sufficiently for about four iterations now (we have each iterations spanning 2 weeks). First the Positive impressions about Scrum: Personally I am impressed by the methodology. Forget everything else, by breaking down the deliverables into atomic units, it helps the managers and engineers to monitor the work at on a regular basis (every two weeks) and make appropriate course corrections whenever mandated. This according to me is the biggest bang for the buck. It builds a healthy working relationships between the different stakeholder...

From a collection of writings I wrote in 2007 when I first explored this methodology . We are almost two iterations old with this process now and many issues have started surfacing up. Listed below are some of the very common comments I have been hearing over last few weeks. I am planning to collect all the comments (will keep updating this post as I hear new ones) and finally intend to check whether the comments are resulting out of Scrum or will result irrespective of the process. I am frustrated with Scrum. It is not working for me, I am not able to make any progress I am irritated with the number of meetings I am attending. I dont think the meetings are helping my productivity I am forced to sit at a stretch for 3 -4 hours. The discussions during these meetings are more of a dialog. I think we are all wasting time with these meetings. What scrum, we still have the same problems between the Development and QA. Development is not giving us the feature till the las...

From a collection of writings I wrote in 2007 when I first explored this methodology . Yesterday one of our Scrum Masters made a comment "What can I do? I have nothing to say. For all practical purpose I am a dummy and it is the team which is responsible". I thought that that was a very stupid comment to make and at the same time was puzzled at that opinion. The textbook definition of a Scrum master is a " person who ensures that the Scrum process is used as intended. The Scrum Master is the enforcer of rules and sprints of practice. The master protects the scrum team from impediments and distractions ". To me this role is quite powerful albeit that unlike a typical project manager, the scrum master does not enjoy the directive controls over the team. But played well, he has lot of leverage to influence the team, lead the team towards right practices and insulate the teams from external distractions. The closest analogy I can think for this role is that of ...

From a collection of writings I wrote in 2007 when I first explored this methodology . Retrospective is a process where in the team reflects upon the just finished iteration and enumerates what went well, what did not go well, what they should continue to be doing, what they should immediately stop doing, and what new things they should start to improve the process. In my view, in order to do justice to this exercise, the following pre-requisites have to be met: The environment should be created such that, people can open up to air their views The primary focus should be internal (i.e. within the team) with the other stakeholders, being good listeners. They effort should be to understand the views/issues raised and help the team in addressing the issues by providing necessary support and guidance. The scrum master should be an active facilitator who will drive the team towards holistic improvements and ensure that the meeting does not reduce to a forum for finding faults. ...

From a collection of writings I wrote in 2007 when I first explored this methodology . Yesterday we completed the first iteration and had the scrum review. Given that this was the first iteration, there was sufficient excitement around here about the demos and how we go about it. I am capturing as part of this blog some of my impressions about this ritual of Scrum. Positives: I personally found that Demos were a real good tool for it gave a perspective of what is coming. Since the audience of the demo included people from other teams, it helped them build a more holistic view of the entire development and helped people to spot dependencies. For many of the young developers, the idea of giving a demo to a large audience was a experience and they were very excited about the same. I personally believe that such excitement is of great importance for the team morale. The demos helped in soliciting quick feedback and make the necessary changes before it is too late. Negatives: ...

From a collection of writings I wrote in 2007 when I first explored this methodology . Last week we had the release planning process for our next release. Given that this is going to be the first release that would be developed in agile, there was lot of expectations on the how to and what to do for release planning. The general idea was to get everyone who would be the primary stake holders for the release (which ironically ends up being senior leads, managers and architects, and product managers) in a single room and trash out the skeleton for the next release. So we had people flown in from US for a week, to be with us in a room to trash out the release contents. Listed below are my observations from the release planning experience. The Process - Get the backlog out - It should list a set of items which is desirable for the release and the priority of the same - Have some offline discussions and research to determine what is involved for each of the item - Give a approxim...

From a collection of writings I wrote in 2007 when I first explored this methodology . Do Managers have a role in the Scrum Process? This was the question that was raised when we were discussing the Scrum Process yesterday. When this question was asked, I had no concrete answers. So today spent some time checking out the web for clues to answer the same. This link (attributed to Jeff)  addresses this question. Reading through that, it appears that in the Scrum world, the role of a “First line Manager” is morphed to that of a mentor and coach, with primary responsibilities like Provides organizational vision Removes impediments ( a role shared with the Scrum Master) Assists with individual development Challenges team beyond mediocrity while respecting team boundaries Helps individuals without sucking the responsibility out of the team Provides the right environment Gives individuals tools to be a great team member Coaches teams through conflict resolution Advo...

From a collection of writings I wrote when I first got exposed to this methodology around 2007 . At the workplace, there has been a sudden impetus to adopt this methodology. Whether we like it or not, we would eventually have to adopt this as there is a tremendous push from the top. Hence have been reading up some stuff and trying to relate it to apply it to my daily professional way of work. From what I have observed, I believe the methodology is promising. It would invariably help in the kind of work we do. Some of the inherent positives I see include: The focus on the working software over extensive documentation  - Most of the time documentation is out of sync with the software and is created more as a ritual than as a knowledge base. The incremental Development approach  - Will provide better control and tracking of the projects. The collective responsibility on the team  - I expect the peer pressures to build accountability to the members of the team and...

Are all Clouds the same.  No. Actually Cloud computing  is an umbrella term that is used to indicate a set of services provided over the net.  At a broad level, the services can be grouped under the following head a) IAAS - Infrastructure as a Service b) PAAS - Platform as a Service c) SAAS - Software as a Service. There are other groupings like DAAS for Data as a Service, MAAS, Monitoring as a Service, etc. But the three mentioned above cover the significant chunk of the offerings currently available on the Cloud.  In this blog, I do not intend to go about defining these all over again.  One can just google on these titles and should be able to get detailed information on each of these.  What I intend to provide is few images which I picked over the net ( I have referenced the source for each of the images) that provide a quick snapshot on what each of the services offer and its value preposition. So what do these mean? ...