Thursday, October 8, 2009

Murphy’s Law and Randomness

Have you ever had that sinking feeling that "Anything that can go wrong will go wrong." (aka Murphy’s Law)? Unfortunately, I just had such an experience in last few weeks. First, I got an expensive NYC “parking violation” ticket for “blocking the box at intersections” as I was stuck for more than one hour outside Holland Tunnel in a Friday afternoon rush hour. Few days later, my car battery died at age of 2 years 5 months, less than half a year after the warranty expired. Then the mechanics told me that I also need a new water pump and thermostat set. As I was busy getting my car to and from the shop for diagnosis and repairs, driver seat of our remaining working transportation in the house got stuck at an awkward position due to worn-out wiring to the motor. On top of all these, my two years old HP laptop died on me unexpectedly that interrupted my routines and suddenly severed my critical link to the outside world. How much more unlucky can one get? I asked myself.

Haven’t you ever had that exasperated negative feeling that "If anything can go wrong, it will, and it will happen at the worst possible time" (aka Finagle's corollary to Murphy's Law)? How come I am often the unlucky guy who needs to refill the paper for that stupid public copy machine despite its huge storage capacity? Why did it always happen exactly when I had no time to spare? How come my precious demo to my company top echelons ran into glitches despite careful preparation and repeated rehearsals? The only saving grace is that it does not seem to discriminate by rank or gender. Remember the embarrassing botched Windows 98 introduction demo at Comdex Show by Bill Gates?

Many have attributed those and other similar sentiments to Murphy’s Law, first coined in 1952, although one may argue an expression of such a feeling can be traced back to 1841 when an Ohio newspaper published a verse:

I never had a slice of bread,
Particularly large and wide,
That did not fall upon the floor,
And always on the buttered side.

According to the Wikipedia article, like many folklore, there have been several accounts and interpretations and the exact story of Murphy’s Law would never be known for sure. Edward Murphy is did exist however. Apparently he was an aerospace engineer who worked on a rocket sled project during late 40’s for U.S. Air Force to measure the G-force during deceleration that a pilot would experience. The legend has it that Mr. Murphy was so disgusted and made comments like “If it can happen, it will happen” when he found out that experiment failed because every wirings of the instrument was wrong and was reversed.

The spread and popularity of Murphy’s Law is probably due to how easily one can relate it to one’s personal daily life and can color it in so many interesting ways. There have been many corollaries and variants of Murphy’s Law and books were compiled and written about them. In particular the most well-known fallacy is the pessimist’s version that “because something (bad) could happen, it is inevitable that it will happen”. I suppose we all long for some explanations to why something happens and why “I am so unlucky”. But are we truly that unlucky sometimes? Or is it actually random and perfectly normal to have a burst of bad events?

Some of those feelings are easier to explain than others. One problem is we sometimes apply empirical data or experiences obtained from a totally different set of observations and vintage points. For example, we tend to relate rush hour traffic reports in terms of usual experience of waiting in checkout lines in stores when the characteristics of the two are totally different. Have you ever wondered why you almost never experienced delays in the order of what traffic reports gave you a short time ago?

Some misguided expectations are a lot more subtle and related to psychology. For instance, there is the well-known hindsight bias. A good example is those incredible stories of how good some fortunate tellers are; few “predictions” that did occur are so much more vivid in one's mind than those that did not. Confirmation bias certainly would just make it worse for those who are ready to buy into the story.

The most intriguing and the real core issuse, I believe, lies with the fact that vast majority of us are very poor judges of what is random and what is not (does this also explain why some of the stories and beliefs in religion played such prominent parts of various societies?). Here is a story to illustrate my point.

The brilliant Math Professor Theodore P. Hill of Georgia Institute of Technology used to ask his students do an assignment at home. The students could either flip a coin 200 times and record the outcomes, or they could choose to fake the 200 outcomes and report the made-up results. You can imagine some “lazy” kids did make up the outcomes and handed the results in. What impressed the students (and Tax Agencies!) was that Professor Hill can almost always identify correctly which results were faked.

His detection magic is based on the observation that one tends to underestimate significantly the likelihood of the occurrence of certain events that may sound or appear to be rare but is not. In the coin tossing experiment described above, the question for those students who choose to fake the results becomes how often and how many heads or tails in a row does he/she expect to appear? It turns out that since the calculation for such a probability of outcomes is non-trivial, most students, when faking the outcomes without knowing what to expect exactly, end up avoiding having sequences of too many consecutive heads or tails that gave away the secret. If one had bothered to work out the details and computation, it turns out that even for a 100 coin tosses, there is almost a 1 in 3 chances that a consecutive 7 heads or tails will appear. That is much higher than what most people would have guessed.

This example shed insights and psychological light into my “why me” problem. I am pretty sure there is no conspiracy nor deliberate forces that made them happen to me in such a short period of time. To his credit as a good engineer, Edward Murphy learned his lessons and advocated Defensive Design since as a result of the failed experiment in his project. My take away of my bad lucks is we all need to do more Defensive Living and make sure we do what we can to guard against what we thought was rare or unlikely random event. Just remember, it is more likely to happen than you think!

Talk to you soon!

No comments: