Are YOU Game?

... and the Lord said, 'let their be light...' yeah and in that moment of enlightenment I could now very clearly see some anti-patterns in my behavior. Let us try and understand the same. Let's say you are pretty very well educated, you have a decent job, you are considered a 'potential' candidate to handle most challenging assignments in your organization and you dream big. But.. and a big one there to be precise, more often than not, you hit a big wall somewhere along the way. You can get something going, but it does not go very far. We have no idea how many times we end up stuck in second gear for the rest of our lives. While we may be “good enough” we could be so much better.

Yes, I wanted to talk about 'Scalability' as much as this concept is borrowed from Software Engineering I plan to adopt it in my day to day life.

"Scalability is a desirable property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner, or to be readily enlarged.[1] For example, it can refer to the capability of a system to increase total throughput under an increased load when resources (typically hardware) are added. An analogous meaning is implied when the word is used in a commercial context, where scalability of a company implies that the underlying business model offers the potential for economic growth within the company.

An algorithm, design, networking protocol, program, or other system is said to scale if it is suitably efficient and practical when applied to large situations (e.g. a large input data set or large number of participating nodes in the case of a distributed system). If the design fails when the quantity increases then it does not scale." - Source: wikipedia

Now scalability has multiple facets to understand;

• 'Scale Out' - Diversify your horizon of understanding, often I have observed that concepts from seemingly unrelated subjects, an example in case the recent rage with Web 2.0 where concepts from Sociology and Technology is gainfully employed to conduct 'Viral Marketing' for businesses. "Tell-A-Friend" widget from Pramati which you see on my blog is niche area platform product operating in this space. It definitely helps to find/create new opportunities if you can apply your diverse knowledge to add value to the society at large. Another way to scale out is to play multiple roles in your team to gain first hand understanding of issues at hand. It may be an enriching experience if you can mix-match-employ your experience with solving engineering problems and handling customer situations.

• 'Scale Up' - Based upon your analysis of the situation you feel it just right to add extra effort.. walk that extra mile to achieve 'customer delight'. [here, customer is overloaded for both the internal and external customer behavior]

• 'Scale Down' - Last but not the least, it is important for us to identify and 'scale down' things which are not working. Things which might become too big a problem while one is trying to scale...
  

With that understanding of issues with me.. I want to ask myself if I'm Game enough to leave my footprint on the third rock from Sun? Game enough to tackle the challenging technology? Game enough to get my game on the planet. I believe I can.. Come on! double up now... faster, higher, stronger.. be a sport :)

Why slow?

Hmm.. so good that you are able to monitor response times of the application components but then how does that helps tuning the application... do not count activity.. show me the productivity.. so here I'm able to monitor my code and generate some couple of reports.. but then they didn't really helped me understand performance bottlenecks.. I would run them again and again to aggregate more reports!! No amount of diligence seemed to work.. imagine I was not able to find problems.. that reminds me of the adage 'bin mange moti mile aur mange mile na bheekh' ... I had understood there is sure shot knowledge gap from technical and business perspective... after all we are still far from creating tools which would scan and identify and fix code on their own. There are just so many options for every refactoring which one intends o make in the code.. one needs to weigh your options before making code change les it should fall under its own weight... Ok, I digressed because even while conducting my own research I chanced to read interesing articles which would give me a peek ino the future about gainfully utlizing concepts from autonomic computing to application monitoring as an aid to debugging applications... but 'Show me the results' was driving me crazy.. but then as the luck would have it a visit to the the local book shop helped me discover "Bitter Java" and as prophecized by "The Alchemist" when you one truly starts liking something the entire universe conspires to make it happen.. seemed so much true .. he "bitter ejb" chapter seems to have been written just for me a quick read with reports immediately helped me identify the 'chatty' interface between our application and the database.. the server frequent roundtrips are consuming far more time over the wire than conducting any productive work... steriotypically such problems occur when we make far more granular requests than is required or may be executing queries in loops... it helps to find initial symptoms of the problem at the macro level.. such an information can be suitably used in other low performing use cases.. bugs come and go, but I'm accumulating debugging skills ... I'm already feeling the need to study "Refactoring: Martin Fowler".. may be I should get back to reading.. more later

I love Johnny Walker...

I love it for the simple reason that it provides me with one of the very core ideology I have decided to follow in my life. It helps me take rough with the smooth and always keeps me brimming with confidence. It helps me overcome difficulties and avoid my indulgence with petty attractions. It helps me take change in my stride. It helps me keep walking...

Performance Monitoring Utility

My current assignment required me to create a light weight utility to conduct application monitoring during production and development phases to identify potential performance bottlenecks. The utility which I created is now ready and is under Beta. Key rationale to create this utility were;

1. Create an unobtrusive (or minimal) way of monitoring the application, although in its current state it does not uses aspect technologies and thus requires us to manually inject code to those applicaion components which we want to instrument.
2. The utility should be real lightweight so that it could be used at all times to monitor application performance and system health.
3. Simple to use. Client code should be able to use the utility in an easy to use way. Figuratively speaking, just place your probes and desired parts of your application and assign a logically comprehend able name to it, that's all!
4. User should be able set monitoring preferences.
5. Reporting monitoring results.
6. Currently the utility monitors the execution times alone but can easily be extended to monitor other metrics like memory, cpu etc.

Design Considerations:

To improve simplicity, reduce verbosity and hide inner working of the monitor code a facade to the monitor library is provided so all hat client code needs to do is;

public SomeObject theMethodUnderInstrumentation(){
try{
MonitorFacade facade = MonitorFactory.getFacade();
if(facade.isMonitorEnabled()){
facade.recordExecutionStart("someLogicalContextName");
}
// do some time consuming task here
}finally{
if(facade.isMonitorEnabled()){
facade.recordExecutionStop("someLogicalContextName");
}
}
}

This is all that is required by the developer to instrument it's code. Simple isn't it!! The above becomes transparent to the developer in case we use aspects to define our pointcuts to inject a similar code as above.

It is a common observation that application performance tuning exercise requires an inter disciplinary approach and much is needed to understand the system performance in the right context. It was therefore important to understand the complete execution path under observation. To make this possible Composite pattern is implemented. The intent of composite is to compose objects into tree structures to represent part-whole hierarchies. The call tree is captured which displays the function execution paths that were traversed in the profiled application. The root of the tree is the entry point into the application or the component. Each function node lists all the functions it called and performance data about those function calls. So to put it crudely composite pattern helped me create the tree structure of application components under observation.

Now that we are ready with the basic infrastructure needed to monitor he code performance at all times the next piece of challenge is to create automated load test and as any one with similar experience will immediately identify that creating automated test cases is not that big a deal but maintaining them over a period of time is.. my next task involves around ways to put intelligence into our load test scripts... I woke up pretty early today .... ;)