बूँद फिर मोती बने ...

एक बूँद 
------------
ज्यों निकल कर बादलों की गोद से
थी अभी एक बूँद कुछ आगे बढ़ी
सोचने फिर फिर यही जी में लगी
हाय क्यों घर छोड़ कर मैं यों बढ़ी
मैं बचूँगी या मिलूँगी धूल में
चू पड़ूँगी या कमल के फूल में
बह गयी उस काल एक ऐसी हवा
वो समन्दर ओर आयी अनमनी
एक सुन्दर सीप का मुँह था खुला
वो उसी में जा गिरी मोती बनी

लोग यौं ही हैं झिझकते सोचते
जबकि उनको छोड़ना पड़ता है घर
किन्तु घर का छोड़ना अक्सर उन्हें
बूँद लौं कुछ और ही देता है कर !

-अयोध्या सिंह उपाध्याय 'हरिऔध`

A Dedication

हम दीवानों की क्या हस्ती

हम दीवानों की क्या हस्ती,
आज यहाँ कल वहाँ चले
मस्ती का आलम साथ चला,
हम धूल उड़ाते जहाँ चले

आए बनकर उल्लास कभी,
आँसू बनकर बह चले अभी
सब कहते ही रह गए,
अरे तुम कैसे आए, कहाँ चले
किस ओर चले? मत ये पूछो,
बस चलना है इसलिए चले

जग से उसका कुछ लिए चले,
जग को अपना कुछ दिए चले
दो बात कहीं, दो बात सुनी,
कुछ हँसे और फिर कुछ रोए
छक कर सुख दुःख के घूँटों को,
हम एक भाव से पिए चले

हम भिखमंगों की दुनिया में,
स्वछन्द लुटाकर प्यार चले
हम एक निशानी उर पर,
ले असफलता का भार चले

हम मान रहित, अपमान रहित,
जी भर कर खुलकर खेल चुके
हम हँसते हँसते आज यहाँ,
प्राणों की बाजी हार चले

अब अपना और पराया क्या,
आबाद रहें रुकने वाले
हम स्वयं बंधे थे, और स्वयं,
हम अपने बन्धन तोड़ चले

- भगवतीचरण वर्मा  

The above poem pretty much summarizes the way I wish I could live my life... trying each moment... keep walking

How about teaching to learn?

Since, my childhood days I was always told that sharing knowledge improves your learning. However, our social conditioning is such that we are not really comfortable to share primarily because;

• Fear of being dispensable, because there are others who may replace you.. is the single most important reason for people to 'hoard' knowledge.
• Fear of being exposed, as sharing might expose your ignorance. 
• ..
• ....

There could be many more, the idea here is not to create an exhaustive list of reasons for people to escape ...  the idea here is to make a public assertion that I no longer want to be a passive consumer of information but create information, distil it for people around me find it easy to consume and add greater value to make the world a better place to live.

Hope, that makes a true dedication to my Mother on this day. I seek your blessings.

Ubuntu 9.10: Wifi connection fix

You have trouble connecting to the wireless network on UBuntu 9.10 try this

sudo apt-get install --reinstall bcmwl-kernel-source

 Hope that helps. It works for me now.

Cascading: How does cascading decides which fields should go to a column family?

I was playing with Cascading code sample as given here.

Problem Statement: let's say we have three fields in a tuple for e.g.

line_num, lower, upper, double
1, a, A, AA

and I wish to add double to its own column family or lets say club it with an existing column family 'right' How do I do that.

Solution:

String tableName = "DataLoadTable";
        Fields keyFields = new Fields("line_num");

// add a new family name
        String[] familyNames = new String[] { "left", "right", "double" };

// group your fields together in the order in which you would like them to be 

// added to column families
        Fields[] valueFields = new Fields[] { new Fields("lower"),
                new Fields("upper"), new Fields("double") };
        HBaseScheme hbaseScheme = new HBaseScheme(keyFields, familyNames,
                valueFields);
        Tap sink = new HBaseTap(tableName, hbaseScheme, SinkMode.REPLACE);

// describe your tuple entry: add the new field
        Fields fieldDeclaration = new Fields("line_num", "lower", "upper",
                "double");
        Function function = new RegexSplitter(fieldDeclaration, ", ");

The remaining of the code remains the same as given in the example.

Either, the above was too obvious that the authors didn't talked about it in the user guide or I do not know how to describe the problem and hence was not able to find them.

Let me know if I'm wrong.

Push Button Automation for the Humans

Have you ever been responsible for the installation of a software which should be distributed and installed on myriad set of execution environments with equally diverse sets of configuration in a department where machines are constantly being cleaned and re-imaged? Picture a situation where you are supporting various configurations and levels of your module. Rather than spending hours installing by hand, wouldn't it be great to have a way to automate the installation process so that you could just kick it off, go and get coffee, come back, and have it all installed and ready? Call it 'Push Button' automation :) (Please bear with me for throwing newer phrases)

Strange as it might seem but this hand-made configuration is proving to be a nightmare of sorts.. one might feel exhausted with out really having any sense of accomplishment by solving petty issues which should not really have come up. High time one must set up;

• standardized installation set up
• provide application diagnostics which must enable operations team to solve small issues in time.

Time again Apache ANT comes to my rescue.

How does the data flows when a job is submitted to Hadoop?

Based on the discussion here, typically the data flow is like this:

1. Client submits a job description to the JobTracker. 
2. JobTracker figures out block locations for the input file(s) by talking to HDFS NameNode. 
3. JobTracker creates a job description file in HDFS which will be read by the nodes to copy over the job's code etc. 
4. JobTracker starts map tasks on the slaves (TaskTrackers) with the appropriate data blocks. 
5. After running, maps create intermediate output files on those slaves. These are not in HDFS, they're in some temporary storage used by MapReduce. 
6. JobTracker starts reduces on a series of slaves, which copy over the appropriate map outputs, apply the reduce function, and write the outputs to HDFS (one output file per reducer). 
7. Some logs for the job may also be put into HDFS by the JobTracker.

However, there is a big caveat, which is that the map and reduce tasks run arbitrary code. It is not unusual to have a map that opens a second HDFS file to read some information (e.g. for doing a join of a small table against a big file). If you use Hadoop Streaming or Pipes to write a job in Python, Ruby, C, etc, then you are launching arbitrary processes which may also access external resources in this manner. Some people also read/write to DBs (e.g. MySQL) from their tasks.

Rampyari ki khoj

What's the big deal, one might ask? Yes, I'm looking for a car and while hunting around I found a wealth of information is available about brands and products available in the market, but one finds zero information about the dealers in the local market.

Market is full of inexperienced, uneducated, incompetent salesmen who are ready grok 'the best' car to you in no time without understanding my needs or expectations. They simply seem to be oblivious about the competition and live a ,monopolistic universe of their own brand, to such an extent that a mere mention about the competition makes them turn blue.

Further, talking to old timers, I realized that your dream car may soon become a nightmare of sorts based upon your service providers, about which you may not have any reliable information. After all any machine needs skilled maintenance and I need some one whom I can trust for the pre-defined level of quality of service in lieu of the amount I spend on the same. The need for rating dealers and service stations is both timely and important.

In the mean while in order to live in the more 'real' world I soon would have to make my choices...

Evolving programming paradigms

The free lunch is over and it simply means that;

• Our applications may no longer be able to automatically seek benefits of hardware up-gradations, we will have to plan, design and make room for concurrency.
• As a developer we need to hone our skills to make good use of hyper-threading, multi-core CPUs and caching.
• Do not get confused with marketing slogans about new found technologies. Seldom, does a technology becomes mature too fast to become main-stream, generally it is old technologies which increasingly useful with time and better understanding of its developers that can gainfully be used to solve important problems.
• Because, our applications do not automatically gain performance with newer hardware solving performance problems and performance tuning will become an important activity.
• Start preparing now.

Urban Area: Community or a Marketplace

'Mumbai for Mumbaikars' might bring bad taste to your mouth or a sense of ownership based upon your political affiliations. Whatever be your reaction it is however increasingly becoming important for us to delve deeper to try and understand the core issues lest we must reject them as pure nuisance.

From a pure theoretical perspective the urban population can be broadly divided into two sections;

• Community dwellers, who inhabits a particular place and encourage symbiotic relationship between various members of the community. They largely like to take the ownership of the place for its general well being and associated identity. Such people have high tolerance towards problems as result of scarcity of resources.
• Market dwellers, who look at an urban area as a place where market operates and they themselves form links into the demand and supply chains. Such people have high preference for quality of life.

The stated difference in their value system creates competition resulting in clashes.

What can be done about it? well .. I'm not too sure will it help if market dwellers contribute back to the community of which they are part off to avoid being  tagged as parasites and community ensures that market operates in safety  thus creating  a conducive environment to attract better talent!!

Part of the answer also requires us to understand the larger picture and avoid such local focus. Will it not help if we have more balanced growth in .. sounds wishful.

Share your mind, who knows our ideas are heard and we do reach a resolution.. let us start from ground zero again..

Mar Java, Mit Java..

... and with European Commission's nod the last hope of keeping the Sun shining too went away, while the whole software and business community is still evaluating the takeover deal, I must express that I feel bad about the whole thing.. there is something which is not generating positive vibes... although, some of you might discount me for being emotional, I have at least one objective reason to support my point in case. Old timers will clearly be able to spot the difference that post acquisition all the Sun domains have been distastefully defaced and simply reflects the tyranny, greed and callous attitude of the raiders.

I, seriously hope similar fate is not vetted out to the open source initiatives.

In bad taste...

Reality television is truly a reality today and one can no longer ignore them. There popularity can be attributed primarily because it also involves its viewers. I used to love 'Boogie-Woogie' for their plain ingenuity and honesty in bringing recognition to dancing talent. The show was conducted wonderfully well.

With time however, the production value of reality shows have increased dramatically but I'm sorry to say that they leave everyone demoralized. Particularly shows like Roadies and Splitsvilla are truly disgusting.

My whole point is that such shows are increasingly creating a 'public acceptance' for foul and abusive language in public which is one very dangerous trend. They must mend their ways or else criminal action be taken against them.

Running Java in debug mode using Apache Ant

Configure your Java task as given below;

<java classname="org.zero.Main" fork="true" failonerror="true">
            <sysproperty key="DEBUG" value="true" />
            <jvmarg value="-Xdebug" />
            <jvmarg value="-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8999" />
            <jvmarg value="-Xnoagent" />
            <jvmarg value="-Djava.compiler=none" />
            <arg value="${project.root}/conf/preferences.xml" />
            <classpath refid="project.classpath" />
        </java>

Further,  configure your eclipse debug remote application to listen to the specified port (8999 in this case).

Also, add Thread.sleep(1000); as the first line in your main method.

Execute Java task and try connecting from eclipse, you may need to increase the thread sleep time.