Who created God?

Who created God? or Is it our own invention? It is an age-old question that has plagued all those who like to think about the big questions.

Just last fortnight, I visited Puri, Orissa for the pilgrimage to Jagannath temple. As much as it was spiritually "satisfying" for my parents I was irritated with the chaos and mayhem created by 'pandas' there. I was wondering if indeed visitors are able to offer their prayers in peace. It is after all every Hindus long cherished wish to do their 'char-dham-yatra' but was that the only purpose or the reason lies in creating another money spinning business in the name of God? Is God our own creation? may be I don't know and neither do I have the patience or time or inclination or competence to read and understand the philosphical content of all the books on the subject. But, here was this craftsman on the Grand Road - famous for the 'Rathyatra' conducted here every year, focussed on giving final touches to idols. His frentic fingure movement was a feast to watch and I was still wondering if he created the God!! Shy of me taking his photos he asked me 'Babu, humari photo kyun le rahe ho?' I told him, I never saw any one creating God to which he replies 'Babu, murti nahin banayenge to ghar kaise chalayenge aur bhagwan to haathon mein hai to khana khila raha hai.. humara to bass itna hi kaam hai'

Soon I realised why do we say 'Apna haath Jagannath'. I guess it is very important to be focussed and do our duty religiously.. rest shall follow. I guess I learn some thing from him.

Maya tumko itna bhi samajh mein nahin aaya!!

Dalit ki beti has done it again... this time she harps back on Manuwadis for targeting her for being partial.. why is it that congress is not targeted for creating 'brand' by naming every little thing after Nehru-Gandhi family member... Read more

I believe instead of crying foul, she can take a lesson here and help creating lets say an IT park in the name of Dr. Ambedkar (hmm.. that will not help BSP) or Kansiram (hmm.. that will help her) Maya Pragat Sangadak Sansthan etc as a matter of fact it doesn't matter if you are trying to bring token benefit for your party provided it brings considerable tangible benefit to people and help them improve their living... get creative Maya you can earn much more by helping people create wealth.. and you can really be in good history books.

Finally spotted the 'Tiger'

Once upon a time long, long, long ago SCJP 5 was newly released and it was a proud moment in my life to secure a distinction in the same. I felt like I 'Tamed the Tiger' never ask how did I feel working on versions 1.4 and 1.3! all these years. Never mind, the 'Tiger' will soon be a sunset technology from Sun's stable... never mind 'Mustang' is galloping faster and faster... Today I spotted the 'Tiger'. I'm happy with the opportunities at hand to experiment... looking forward to play with the beast...

School chalein hum...

Coming from a modest middle-class background I understand and appreciate the importance of education in securing one's future. Not that people from other background don't appreciate education or that careers in other fields do not exist. But, pragmatically speaking education can help you achieve sustainable living and it is crucial in managing one's self esteem, specially for those from related backgrounds. Probably, that sets the pretext for the heightened stress that parents and children undergo to secure better grades, which has even led to suicides etc in some really unfortunate cases.

The recent education policy aims to abolish school board exams quoting the need to remove [reduce?] the stress for school going children. A welcome step? Well, depends where are you coming from.. In case you feel it is a welcome step because it help reduce stress I'm not with you. My question is who are you trying to help? are you saying you can make their life a bed of roses? they are not going to have any stress? ever? May be your new system allows them to pass certain public exam 'without failing' but what next? are they going to spend their lives at school alone, studying? I believe, No.. some time or the other they need to face the industry, the real world.. believe me all that school work will look like a cake walk...

But, in case you do believe that abolishing public exams at school level is welcome step because now students do not have to study to get grades. Yes, as much as it sounds strange it is good. It is important to understand that education is not some material object [read grades/marks] that one gets upon completion of education. The real education is the state of mind. It is that great heritage of common sense that has been brought down to us over the ages. Now, with that material motivation removed students can now focus on learning rather than getting good grades, which makes them seeker. The motivation to learn is now from within, there are no socially defined pressures.

I understand what many of us might go through as a result of this void created in our lives. There are no 'goals' which might distract our 'normal' student and he may eventually loose his interest in studies and compromises on a lesser education ... which is good isn't it?!! think about it!!... he was not liking it any ways and at best would have remained a mediocre all his life even after a 'highly-rated-degree'. Only those who really appreciate their learning will continue thus saving a lot of resources.

In either case there are certain risk factors which need to be mitigated to avoid chaos:

• The 'normal' student who settled for a lesser education as a result of his ever increasing committment to his 'other-than-education' activities should be able to leave his education without any penalties
• He should be able to find an employment opportunity fit to his calibre.
• He should be able to return to his education in case he gets frustrated with his 'boring' work to seek creativity to gain theoretical understanding of THE World around him.

Learning should be fun.

Imagining Female Foeticide and Gay marriages and Prostitution

All along I was tagged as a happy-go-lucky person who would go out of his way to have fun.. for that one moment of 'nirmal anand' and the best compliment to me was when some called me a 'stress buster' and further asked what's your source of energy? how do you do that? In reply to which I would say I'm happy as I'm gay!!! and then everyone would burst out to laughter.. I dare say that again and people would take it really seariously .. thanks to the recent turn of events in our society. Recent court judgements have 'legalised' the gay/lesbian marriages leading to nationwide celebrations for their newly acquired "freedom". I was wondering what's the real cause for their inclination to the unantural carnalness... Is that their utmost frustration in finding the right girl? Is it because of acute gender imbalance in their community? Is it THE result of female foeticide in our country? I mean I'm also not in favor of moral poilicing but then isn't it banal and disgusting or may be I don't understand... so what's next? Let's legalize prostitution and earn more money for governement coffers..

The whole point is such act of recognition will only encourage impressionable minds to follow the next 'IN' thing to prove them to be 'COOL' and the by the time they will realize their mistakes it will be to late...

Complete idiot's guide to Debugging

Time and again I have learnt much to my displeasure that I'm not efficient while debugging. Most often than otherwise, I would just miss the point and keep 'searching' for the issues. Sitting back I found one fundamental flaw in my approach to debugging a problem that there was no structure in my method! Instead of narrowing the cause-effect scenarios I was simply 'searching for the needle in the hay stack'. This would obviously take more time than expected and would lead to sheer wastage of my resources.

Learning about the scientific method of experimentation I found that solution of problems too complex for common sense to solve are achieved by mixture of inductive and deductive techniques.

The inductive logic demands us to start observing the 'system' and arrive at general conclusions. Make a mental note of your observations and try finding explanations to them chances are you would hit few observations which are not clearly understood in the given context, some thing which is kind of a 'mystery' and you find it tough to explain it the first time, hold on and do not get distracted collect information. Having done that now try to break your thoughts and write them under following heads;

1. Problem statement: The biggest mistake to avoid here is to write too much, things which you are not sure of.. just an assumption. Rather, it is much better to state the problem straight. It might sound dumb in he beginning but would save from making dumb efforts based on incorrect assumptions later ..
2. Hypotheses to he problem; typically list down possible reasons for the failure
3. Experiments to test your hypotheses: The most important mistake to avoid here is that the experiments must be designed to test the hypotheses nothing more nothing less. This step would help you narrow down your solution/problem area.
4. Expected outcomes will help you draw a base line for your experimentations.
5. Actual outcomes must be compared and contrasted with the expected outcome for rigour in the testing process. Any deviations must be carefully studied and isolated to ensure that the anomaly is not because of some mistakes/incorrect measurements this will save one from drawing incorrect conclusions.
6. Conclusion, here again care must be taken to state no more than the experiment proves/disproves.

By asking the right questions, conducting the right tests and drawing the right conclusions one might drill down the hierarchical structure of the problem to isolate the point of failure and fix it so that the system no longer fails at this point.

One might need to-fro between the two logical techniques to reach a plausible solution. Need to put this learning in practice now...

Debugging

This is funny, this is not funny for me.

During this debugging session, I'm continuously guilt ridden for the technical failure is delaying the delivery and it was increasingly time consuming because I needed to question each of my assumptions, that sucks!

After my stress-profile-code-profile-stress cycles I did however learned a bit more about various JVM GC tuning options. Also, that in an overzealous attempt to gain customer delight the code was doing far more than required. I moved a step back and thinking from the usability perspective the problem immediately visible. Thanks to right encapsulation none of the interfaces are breaking and the affected code module could be refactored to fix the performance issue at hand. Further, tweaking the data structure to collect data for generating the performance reports helped us achieve performance gains. Now, we are able to do more with less. The trick is finding the application of technology that makes the most sense for solving your problem at hand.

Looks like my build is complete, let me profile my code again..

Java: Heap crunch

Profile-code-deploy-load was all that I did yesterday and yes it did improve the system performance and technically it did survived the brutal four hour load test without causing OOOE (out of memory exception). The primary hypothesis under which I'm working is that their the certain long living references have retained a sizable of memory. But, I'm sure I'm not exactly able to find the root cause for the system degradation under load. The important observations are;

• We are loading the system using a pattern where we ramp up the load in x minutes and maintain the peak load for y hours and hen ramp down to zero in x minutes.
• The heap usage percentage follows the similar pattern during ramp up, but although the load has plateaued the heap usage percentages continues go higher, while ramp down it eases a bit but never follows the decrease in load and never actually reaches zero or minimum even though there is no activity in the application.
  
• CPU times in GC increases almost linearly albeit with a low gradient and decreases to zero after load completes.

I plan to study GC logs today to understand the system characteristics and fine tune the JVM fit for our purpose and I am sill not sure if I have a memory leak... more later.

Java: Memory leak

Around two years back, when I encountered an article mentioning memory leak in Java, it was little surprising for me and may be author knew about his audience and went ahead to provide one demo program. It surely made me aware of the issue and certain anti-patterns to avoid. Then, I encountered my very own first Java memory leak, and that I my skills are not adequate to handle them. Here, in the current case, when I first observed this phenomenon I went ahead and explicitly 'nulled' my references, which are other wise supposed to go out of scope any ways and hence eligible for garbage collection, but to my frustration this has only marginally helped as the changes have only reduced the rate of memory leak and has not eliminated it. Now I plan to work to work associated anti-patterns like that weak references or references to small living objects held by long living objects etc. Will keep you posted.

Just like that...

We Indians have a tendency to generalize thing so much so that it hardly takes any effort from our side to paint the world in one color. Often as an extension to this exercise to tend o extract particular characteristics of people from a city, and I quite believe that any self respecting engineer from a decent engineering college in India must have had great fun using them to describe their friends. Almost every where you can find 'Baba', 'Guru', 'Ziddi Jat', 'Sirfira Sardar', 'Bihari', 'Taant', 'Gulti', 'Kannadiga', 'Khatta', 'Ghatan'... and the list goes on and on... while I was siting wih some of my senior friends we happened to discuss the changing times.. the arch typical '... those were the day, ...' and one of my senior friend was quite amused and interested to learn the diversity, he then told us that back then during his times such richness and variety was unheard of and as a result of which he would often be referred to as 'Madrasi', now that was real bad as this could be the worst form of identity crisis... he complained that all those from 'North India' used to refer any one down south with the same term and he would never be able to take that... and in order to have revenge on them he would paint them (Northies) as 'Bihari' and that had far reaching affect on them and upon complain he would highlight the similar difference to them... and that probably sowed the seed to heightened verbosity currently in use... seems like this is what is referred to as evolution of language ;)

Disclaimer: With malice to no one... have fun, n'joy!

Failed to connect to remote VM. Connection refused. Connection refused: connect

A lot of time was wasted to resolve the above mentioned error, while I was trying to connect to a remote server using my eclipse IDE in debug mode. Here is the solution;

1. Compile your application using '-g' option. This will generate all the debugging related information, including local variables. By default, only line number and source file information is generated.
2. Launch a new Java Virtual Machine (VM) with the main class of the application to be debugged. For e.g. Launch a new Java Virtual Machine (VM) with the main class of the application to be debugged. For e.g. $> gt; java -Xdebug -Xnoagent -Xrunjdwp:transport=dt_socket, server=y, suspend=n, address=8000

Hope that helps.

Are YOU Game?

... and the Lord said, 'let their be light...' yeah and in that moment of enlightenment I could now very clearly see some anti-patterns in my behavior. Let us try and understand the same. Let's say you are pretty very well educated, you have a decent job, you are considered a 'potential' candidate to handle most challenging assignments in your organization and you dream big. But.. and a big one there to be precise, more often than not, you hit a big wall somewhere along the way. You can get something going, but it does not go very far. We have no idea how many times we end up stuck in second gear for the rest of our lives. While we may be “good enough” we could be so much better.

Yes, I wanted to talk about 'Scalability' as much as this concept is borrowed from Software Engineering I plan to adopt it in my day to day life.

"Scalability is a desirable property of a system, a network, or a process, which indicates its ability to either handle growing amounts of work in a graceful manner, or to be readily enlarged.[1] For example, it can refer to the capability of a system to increase total throughput under an increased load when resources (typically hardware) are added. An analogous meaning is implied when the word is used in a commercial context, where scalability of a company implies that the underlying business model offers the potential for economic growth within the company.

An algorithm, design, networking protocol, program, or other system is said to scale if it is suitably efficient and practical when applied to large situations (e.g. a large input data set or large number of participating nodes in the case of a distributed system). If the design fails when the quantity increases then it does not scale." - Source: wikipedia

Now scalability has multiple facets to understand;

• 'Scale Out' - Diversify your horizon of understanding, often I have observed that concepts from seemingly unrelated subjects, an example in case the recent rage with Web 2.0 where concepts from Sociology and Technology is gainfully employed to conduct 'Viral Marketing' for businesses. "Tell-A-Friend" widget from Pramati which you see on my blog is niche area platform product operating in this space. It definitely helps to find/create new opportunities if you can apply your diverse knowledge to add value to the society at large. Another way to scale out is to play multiple roles in your team to gain first hand understanding of issues at hand. It may be an enriching experience if you can mix-match-employ your experience with solving engineering problems and handling customer situations.

• 'Scale Up' - Based upon your analysis of the situation you feel it just right to add extra effort.. walk that extra mile to achieve 'customer delight'. [here, customer is overloaded for both the internal and external customer behavior]

• 'Scale Down' - Last but not the least, it is important for us to identify and 'scale down' things which are not working. Things which might become too big a problem while one is trying to scale...
  

With that understanding of issues with me.. I want to ask myself if I'm Game enough to leave my footprint on the third rock from Sun? Game enough to tackle the challenging technology? Game enough to get my game on the planet. I believe I can.. Come on! double up now... faster, higher, stronger.. be a sport :)

Why slow?

Hmm.. so good that you are able to monitor response times of the application components but then how does that helps tuning the application... do not count activity.. show me the productivity.. so here I'm able to monitor my code and generate some couple of reports.. but then they didn't really helped me understand performance bottlenecks.. I would run them again and again to aggregate more reports!! No amount of diligence seemed to work.. imagine I was not able to find problems.. that reminds me of the adage 'bin mange moti mile aur mange mile na bheekh' ... I had understood there is sure shot knowledge gap from technical and business perspective... after all we are still far from creating tools which would scan and identify and fix code on their own. There are just so many options for every refactoring which one intends o make in the code.. one needs to weigh your options before making code change les it should fall under its own weight... Ok, I digressed because even while conducting my own research I chanced to read interesing articles which would give me a peek ino the future about gainfully utlizing concepts from autonomic computing to application monitoring as an aid to debugging applications... but 'Show me the results' was driving me crazy.. but then as the luck would have it a visit to the the local book shop helped me discover "Bitter Java" and as prophecized by "The Alchemist" when you one truly starts liking something the entire universe conspires to make it happen.. seemed so much true .. he "bitter ejb" chapter seems to have been written just for me a quick read with reports immediately helped me identify the 'chatty' interface between our application and the database.. the server frequent roundtrips are consuming far more time over the wire than conducting any productive work... steriotypically such problems occur when we make far more granular requests than is required or may be executing queries in loops... it helps to find initial symptoms of the problem at the macro level.. such an information can be suitably used in other low performing use cases.. bugs come and go, but I'm accumulating debugging skills ... I'm already feeling the need to study "Refactoring: Martin Fowler".. may be I should get back to reading.. more later

I love Johnny Walker...

I love it for the simple reason that it provides me with one of the very core ideology I have decided to follow in my life. It helps me take rough with the smooth and always keeps me brimming with confidence. It helps me overcome difficulties and avoid my indulgence with petty attractions. It helps me take change in my stride. It helps me keep walking...

Performance Monitoring Utility

My current assignment required me to create a light weight utility to conduct application monitoring during production and development phases to identify potential performance bottlenecks. The utility which I created is now ready and is under Beta. Key rationale to create this utility were;

1. Create an unobtrusive (or minimal) way of monitoring the application, although in its current state it does not uses aspect technologies and thus requires us to manually inject code to those applicaion components which we want to instrument.
2. The utility should be real lightweight so that it could be used at all times to monitor application performance and system health.
3. Simple to use. Client code should be able to use the utility in an easy to use way. Figuratively speaking, just place your probes and desired parts of your application and assign a logically comprehend able name to it, that's all!
4. User should be able set monitoring preferences.
5. Reporting monitoring results.
6. Currently the utility monitors the execution times alone but can easily be extended to monitor other metrics like memory, cpu etc.

Design Considerations:

To improve simplicity, reduce verbosity and hide inner working of the monitor code a facade to the monitor library is provided so all hat client code needs to do is;

public SomeObject theMethodUnderInstrumentation(){
try{
MonitorFacade facade = MonitorFactory.getFacade();
if(facade.isMonitorEnabled()){
facade.recordExecutionStart("someLogicalContextName");
}
// do some time consuming task here
}finally{
if(facade.isMonitorEnabled()){
facade.recordExecutionStop("someLogicalContextName");
}
}
}

This is all that is required by the developer to instrument it's code. Simple isn't it!! The above becomes transparent to the developer in case we use aspects to define our pointcuts to inject a similar code as above.

It is a common observation that application performance tuning exercise requires an inter disciplinary approach and much is needed to understand the system performance in the right context. It was therefore important to understand the complete execution path under observation. To make this possible Composite pattern is implemented. The intent of composite is to compose objects into tree structures to represent part-whole hierarchies. The call tree is captured which displays the function execution paths that were traversed in the profiled application. The root of the tree is the entry point into the application or the component. Each function node lists all the functions it called and performance data about those function calls. So to put it crudely composite pattern helped me create the tree structure of application components under observation.

Now that we are ready with the basic infrastructure needed to monitor he code performance at all times the next piece of challenge is to create automated load test and as any one with similar experience will immediately identify that creating automated test cases is not that big a deal but maintaining them over a period of time is.. my next task involves around ways to put intelligence into our load test scripts... I woke up pretty early today .... ;)

Tip: Sort a map by values

A handy tip to sort map by values.

//:~
package samples.core.java;

import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.TreeMap;

public class SortMapByValueExample {
public static void main(String[] args) {
Map map = new TreeMap();
map.put("1", "100");
map.put("10", "50");
map.put("100", "40");
map.put("1000", "10000");

List keys = new ArrayList(map.keySet());
ByValueComparator bvc = new SortMapByValueExample().new ByValueComparator(
map);
Collections.sort(keys, bvc);
Iterator iterator = keys.iterator();
while (iterator.hasNext()) {
String key = (String) iterator.next();
System.out.println(key + "-" + map.get(key));
}
}

private class ByValueComparator implements Comparator {
private Map map;

public ByValueComparator(Map map) {
this.map = map;
}

public int compare(Object o1, Object o2) {
int integer = Integer.parseInt((String) this.map.get((String) o1));
int otherInteger = Integer.parseInt((String) this.map
.get((String) o2));
return -1
* (new Integer(integer)
.compareTo(new Integer(otherInteger)));
}
}
}

Performance monitoring - creating home grown utilities

One of my current focus area is performance engineering, which although is an umbrella concept in itself pretty much sums the need to achieve the non-funcional requirement defined for the software solution. You may find a neat introduction the concept here. As much as it is important from software engineering perspective, talking of it during design session is looked upon as 'over-engineering' and is grossly ignored saying 'It will be one nice problem to solve, but we are not there yet...' probably, I understand the product manager's need to hit the market ASAP.

As they say, 'Dates in the calendar are closer than they appear', we normally start receiving complaints in form of poor user experience, loss of revenue to the competition blah, blah.. so now we have the 'nice' problem actually waiting for us to solve. Frankly, to start with there are no straight answers or atleast I do not have one. What do we do now? Where do we start? Which are the ideal candidates for performance tuning exercise? Here, is what I would do to track the problem. Place an extremely light weight activity monitoring utility in production at all times because I deem it necessary to even capture the usage of the application by the real life users and not us software engineers enmasked as pseudo users [During one of my earlier assignments, the users of the application would simply close the browser window wih out logging out, because the application didn't contained his private data and just that the users didn' like to wait for the application to log out... resulting in numerous open sessions...]

There are plenty of open source and commercially available tools in the market, still if you feel the need to create a home grown solution for some reason, here is the simple idea that I'm working on now to collect metrics from different parts of the application performance and use it for the quantitative analysis of the application.

Provide a ThreadLocal storage to 'StatisticsCollector' which would provide probes to be kept at key parts of the application to record statistics, to start with I'm only collecting execution times. Create a new object each time a new thread is spawned and keep adding metrics for the components with their metrics at collection points and them to the tree data structure which represents the call hierarchy. Finally, once the response is committed for the thread persist the data and reset thread local variables. I have primarily identified following collection points viz. the servlet filter, servlet do*(), JSPs, transactions, cache hits, jdbc layer while they are less in number they can pretty much indicate the slowest moving layer in my application. This information will form the input o iterative process of profiling-tuning-monitoring to gain performance improvement upon completion of each cycle.

Buckle up - Enterprise Edition

Time and again people ask me, if I still think I made the right decision to cut short my 'coveted' foreign assignment to return India because I felt it 'did not add value'... well let time keep tab on that. My story is that... whatever role life gives you play it big [this one is lifted from Smirnoff's ad, we can even learn from a Daru company ;)].

We[Indian IT industry] started real small with back office support work but soon gained confidence of the world to provide medium to large scale system enhancements from where we incrementally we created value to achieve customer delight through our efficiency and professionalism. It is no wonder that today IT captains are trusted partners for very large scale system transformation exercises. In essence, we have incrementally moved up the value chain.

Now, it is important for us to move up to the next step where in we need to conceptualize and develop 'product(s)'. Definitely, this would be a much riskier business. But, then are there any options for us but to grow better. The world will truly respect our technical prowess if we 'create' new concepts and 'implement' them... and can there be better times than these. Here is why I think these are the best of time to go deep...

• Under the current scheme of affairs techies prefer to stay home because much of the quality work will be done off-shore and more importantly an increased focus on domestic projects gives them an opportunity to solve 'their' own problems. This tends to improve employee job satisfaction as it allows them to play 'lead' roles.
• Working on domestic projects will create more avenues where we can find generic 'patterns' to be created as 'products'.
• Investing into products will make employee confident about the longevity of vision of the organization leaders.
• Confidence breeds success.

NASSCOM is equally exhilarated to display the abounding capacity of Indian software companies representing products conceptualized, designed and made in India. It has come up with a logo as displayed above. You may wish to read more here

It is all in mind and we can definitely make a difference by shaping our reactions to the events in life.

Buckle up!

Way back in the year 2002, I witnessed the placements of my super seniors. I remember it took some one month of time for its completion... compare that with ours which got over in a matter of a day or two. In my observation it made them a better human being. I really thought if God could grant me those challenges to me too.

Everyone would agree these are tough times, but then I have a difference of opinion about the way we react to the current state of affairs. There is wide spread fear of failure and confidence index is diving south. Here, is how I plan to handle the situation at hand.

• First earn and then spend.
• Innovate at work.
• Remain competitive.
• Retain your enthusiasm.
• Protect your vision of the future.
  
• Protect your humor, always.