Friday, September 24, 2010

Race etiquettes

Read some them here,
http://runnersforlife.com/page/race-etiquette

Friday, July 23, 2010

Talking common sense

'Common sense is quite uncommon' is what I was told whenever I failed to cut my way through the chaos. As much as, I was frustrated with my failure to learn from my failures. I equally wondered if there is a way to master the technique. Today, I wish to share my learnings with my readers.

'Common sense is a combination of experience, training, humility, wit and intelligence'. Wow! now that sounds like a well balanced equation with known variables. Doesn't that excites you to take control of participating variables and work on your weaker areas to improve your gross wisdom quotient! Let's work on that too;
  • Experience is nothing but the accumulation of knowledge or skill that results from direct participation in events or activities. Simply put, just do it will alone help learn things in small digestible chunks. Fear of failure is the greatest impediment to learning. Make it your friend!! Adapt a fail fast approach. trust me it always helps and at least you end up enriching your experience.
  • Training refers to the acquisition of knowledge, skills, and competencies as a result of the teaching. But, where do I start. Look around yourself. Be relevant to your eco-system. Learning something which is immediately useful keeps you motivated and provides avenues to 'Apply Yourself'. Remember no one learns or appreciates swimming by reading the book!!
  • Pride eats up your brain, being humble means taking good and bad in your strides and accept them as part of our lives. It always helps to balance your emotions [I know I'm being preachy, I need a lot ground work here.. but then whose blog is this ;) ]
  • When in trouble use your humor! It sheds the unnecessary weight from your shoulder and helps you relax. It's very important. Enjoy! every moment.
  • Intelligence is you ability to comprehend, to profit from your experience which now looks like product of the healthy concoction of the above ingredients.. the 'executable knowledge' !!
There you have it! the prescription to work on your common sense. Happy learning. You may now join you hands for a big round of applause... Have fun!!

Tuesday, June 15, 2010

Ride to Kudremukha

It's been so long that I last went out on a ride. Then, there these set of really enthusiastic guys who infected me with the travel bug. Get Off Your Ass!! was the war cry for us and we simply carried our back-packs and hi the road straight .. late night driving, after a really long week was proving to be physically taxing on us and we decided to call it the day at Arasikere, starting Saturday morning we visited Halebidu, Belur on our way and they took a detour to Chikmanglur and then through Ghas we reached Horanadu, little did we knew that there is a majestic temple of Goddess Annapurna. The place was really peaceful, one may just go there and stay over the weekend. It was blissful to walk in the clouds and funny to wach clouds enter our rooms and pour water on our belongings. Trust me we were drenched to core, so much so, that even our spirits must have also satiated their thirsts. We slept like Dogs, no idea when did the dawn broke on us and after one quick shower we headed towards the temple. After seeking the blessings, we hit the road again and after loosing our way to he place we went through the ghats uphill to Kudremukh, KIOCL colony and then to Lakya Dam. It was just plain fun to drive in the rains and watch the sight of nature's beauty along coffee estates, many of which just could not captured, lest we must loose our camera to heavy rains.

On our return trip, we reached Hassan town and then took the road to Bangalore. Journey was quite eventful and we have a whole lot of fun stories to share.. I miss my old gang.. God knows .. kiski nazar lag gayi :|

Keep walking..

Find the pictures here

Maps: To, From

Monday, June 7, 2010

Training for long distance running

Suddenly something! No, way.

I was bitten by this bug during my school days. Just that, I lack discipline and often my initial enthusiasm causes more harm than necessary. Often my strong will to complete certain distance over powers my muscular strength :)

Basically, this marathon thingy is more than just adding miles to my legs, I must admit it is highly rejuvenating, some how I feel very good after half an hour of running. It even helps me fight emotional voids being created in me.

I pledge to complete my Sunfeast half marathon next year.

You may take tips from the references below. Do share your tips.

References:
  • http://www.runnersworld.com/
  • http://marathontraining.com/

Tuesday, May 25, 2010

Hadoop application packaging

Job jar must be packaged as below;

job.jar
|--META-INF
|----MANIFEST.INF
|------Main-Class: x.y.z.Main
|--lib
|---- commons-lang.jar Note: Place your dependent jars inside lib directory
|--org.zero
|---- application classes here

Archiving large number of small files into small number of large files

A small file is one which is significantly smaller than the HDFS block size (default 64MB).

We have a lot of data feeds in the range of 2MB per day, storing each as a separate file is non-optimal.

The problem is that HDFS can't handle lots of files, because, every file, directory and block in HDFS is represented as an object in the namenode's memory, each of which occupies 150 bytes. So for 10 million files, each using a block, would use about 3 gigabytes of memory. Scaling up much beyond this level is a problem with current hardware. Certainly a billion files is not feasible.

Furthermore, HDFS is not geared up to efficiently accessing small files: it is primarily designed for streaming access of large files. Reading through small files normally causes lots of seeks and lots of hopping from datanode to datanode to retrieve each small file, all of which is an inefficient data access pattern.

Also, HDFS does not supports appends (follow http://www.cloudera.com/blog/2009/07/file-appends-in-hdfs/).

Known options are;
  1. Load data to Hbase table and periodically export them to files for long term storage. Some thing like we have product log for a particular date/timestamp against the content of the files stored as plain text in Hbase table.
  2. Alternatively, we can treat these files as pieces of the larger logical file and incrementally consolidate additions to a newer file. That is, file x was archived on day zero, the next day new records are available to be archived. We will rename the existing file to let's say x.bkp and then execute a mapreduce job to read the content from the exiting file and the new file to the file x.
  3. Apache Chukwa solves the similar problem of distributed data collection and archival for log processing. We can also take inspiration from their and provide our custom solution to suit our requirements, if needed.

Saturday, May 22, 2010

One wish

को‌ई गाता मैं सो जाता

संस्कृति के विस्त्रित सागर मे
सपनो कि नौका के अंदर
दुख सुख कि लहरों मे उठ गिर
बहता जाता, मैं सो जाता ।

आँखों मे भरकर प्यार अमर
आशीष हथेली मे भरकर
को‌ई मेरा सिर गोदी मे रख
सहलाता, मैं सो जाता ।

मेरे जीवन का खाराजल
मेरे जीवन का हालाहल
को‌ई अपने स्वर मे मधुमय कर
बरसाता मैं सो जाता ।

को‌ई गाता मैं सो जाता
मैं सो जाता
मैं सो जाता

 - हरिवंशराय बच्चन

बूँद फिर मोती बने ...


एक बूँद 
------------
ज्यों निकल कर बादलों की गोद से
थी अभी एक बूँद कुछ आगे बढ़ी
सोचने फिर फिर यही जी में लगी
हाय क्यों घर छोड़ कर मैं यों बढ़ी
मैं बचूँगी या मिलूँगी धूल में
चू पड़ूँगी या कमल के फूल में
बह गयी उस काल एक ऐसी हवा
वो समन्दर ओर आयी अनमनी
एक सुन्दर सीप का मुँह था खुला
वो उसी में जा गिरी मोती बनी

लोग यौं ही हैं झिझकते सोचते
जबकि उनको छोड़ना पड़ता है घर
किन्तु घर का छोड़ना अक्सर उन्हें
बूँद लौं कुछ और ही देता है कर !

-अयोध्या सिंह उपाध्याय 'हरिऔध`

A Dedication

हम दीवानों की क्या हस्ती

हम दीवानों की क्या हस्ती,
आज यहाँ कल वहाँ चले
मस्ती का आलम साथ चला,
हम धूल उड़ाते जहाँ चले

आए बनकर उल्लास कभी,
आँसू बनकर बह चले अभी
सब कहते ही रह गए,
अरे तुम कैसे आए, कहाँ चले
किस ओर चले? मत ये पूछो,
बस चलना है इसलिए चले

जग से उसका कुछ लिए चले,
जग को अपना कुछ दिए चले
दो बात कहीं, दो बात सुनी,
कुछ हँसे और फिर कुछ रोए
छक कर सुख दुःख के घूँटों को,
हम एक भाव से पिए चले

हम भिखमंगों की दुनिया में,
स्वछन्द लुटाकर प्यार चले
हम एक निशानी उर पर,
ले असफलता का भार चले

हम मान रहित, अपमान रहित,
जी भर कर खुलकर खेल चुके
हम हँसते हँसते आज यहाँ,
प्राणों की बाजी हार चले

अब अपना और पराया क्या,
आबाद रहें रुकने वाले
हम स्वयं बंधे थे, और स्वयं,
हम अपने बन्धन तोड़ चले

- भगवतीचरण वर्मा  

The above poem pretty much summarizes the way I wish I could live my life... trying each moment... keep walking

Sunday, May 9, 2010

How about teaching to learn?


Since, my childhood days I was always told that sharing knowledge improves your learning. However, our social conditioning is such that we are not really comfortable to share primarily because;
  • Fear of being dispensable, because there are others who may replace you.. is the single most important reason for people to 'hoard' knowledge.
  • Fear of being exposed, as sharing might expose your ignorance. 
  • ..
  • ....
There could be many more, the idea here is not to create an exhaustive list of reasons for people to escape ...  the idea here is to make a public assertion that I no longer want to be a passive consumer of information but create information, distil it for people around me find it easy to consume and add greater value to make the world a better place to live.

Hope, that makes a true dedication to my Mother on this day. I seek your blessings.

Wednesday, April 28, 2010

Ubuntu 9.10: Wifi connection fix

You have trouble connecting to the wireless network on UBuntu 9.10 try this
sudo apt-get install --reinstall bcmwl-kernel-source
 Hope that helps. It works for me now.

Sunday, March 28, 2010

Cascading: How does cascading decides which fields should go to a column family?

I was playing with Cascading code sample as given here.

Problem Statement: let's say we have three fields in a tuple for e.g.
line_num, lower, upper, double
1, a, A, AA

and I wish to add double to its own column family or lets say club it with an existing column family 'right' How do I do that.

Solution:
String tableName = "DataLoadTable";
        Fields keyFields = new Fields("line_num");
// add a new family name
        String[] familyNames = new String[] { "left", "right", "double" };
// group your fields together in the order in which you would like them to be 
// added to column families
        Fields[] valueFields = new Fields[] { new Fields("lower"),
                new Fields("upper"), new Fields("double") };
        HBaseScheme hbaseScheme = new HBaseScheme(keyFields, familyNames,
                valueFields);
        Tap sink = new HBaseTap(tableName, hbaseScheme, SinkMode.REPLACE);
// describe your tuple entry: add the new field
        Fields fieldDeclaration = new Fields("line_num", "lower", "upper",
                "double");
        Function function = new RegexSplitter(fieldDeclaration, ", ");

The remaining of the code remains the same as given in the example.

Either, the above was too obvious that the authors didn't talked about it in the user guide or I do not know how to describe the problem and hence was not able to find them.

Let me know if I'm wrong.

Thursday, March 11, 2010

Push Button Automation for the Humans

Have you ever been responsible for the installation of a software which should be distributed and installed on myriad set of execution environments with equally diverse sets of configuration in a department where machines are constantly being cleaned and re-imaged? Picture a situation where you are supporting various configurations and levels of your module. Rather than spending hours installing by hand, wouldn't it be great to have a way to automate the installation process so that you could just kick it off, go and get coffee, come back, and have it all installed and ready? Call it 'Push Button' automation :) (Please bear with me for throwing newer phrases)

Strange as it might seem but this hand-made configuration is proving to be a nightmare of sorts.. one might feel exhausted with out really having any sense of accomplishment by solving petty issues which should not really have come up. High time one must set up;
  • standardized installation set up
  • provide application diagnostics which must enable operations team to solve small issues in time.
Time again Apache ANT comes to my rescue.