Showing posts with label science and technology. Show all posts
Showing posts with label science and technology. Show all posts

Friday, June 20, 2014

Miracles No More?!


When we see or hear about an event that we did not expect it to happen, we react with surprise.  We describe them as rare, unusual, improbable, coincidence, unbelievable, and sometimes outright miracles.  Have you ever felt that these supposedly rare events seem to happen much more often than you expected?  I submit that many of us feel the same.  Further, throughout the human history, people have been seeking explanations why these improbably events happen.

One possible explanation is that many of such events indeed shouldn’t have happened.  Therefore, since they did, they must be caused by the power of some unknown non-human entities or forces.   Another possibility is that our expectation was too low.  We probably have failed to account for some factors and thus significantly underestimated the possibility for those events to occur.  That is, they are not rare events to begin with.  Or it is simply that we haven’t acquired sufficient knowledge and tools to provide adequate explanations.

To address this issue is by no means easy.  Fortunately, with the advances in philosophy, sciences and mathematics, we have already come a long way in clarifying and quantifying the issue rationally and logically.  In fact, there are decent tools now for us to tackle the question if the occurrence of certain events is likely to be truly unusual.

But the harder problem is actually in convincing those who thought an event is a miracle when it is not in the first place.  Part of the challenge is to show the reasoning to laymen who are not familiar with the language and tools of mathematics and statistics.  The landscape has changed earlier this year when David Hand, professor emeritus of Imperial College London and a former President of the (British) Royal Statistics Society, published his beautiful little book The Improbability Principle – why coincidences, miracles, and rare events happen every day. 

As the title indicates, Professor Hand showed us eloquently with many real-life examples and stories how easy and often we get impressed with events/news that we thought should not have happened but did.  He identifies the main culprit what he called Improbability Principle – a collection of strands/laws, each of which is sufficient to produce outcomes that appear to be highly improbable to the uninitiated.  What makes the matter much worse and so difficult to discern is that these strands of Improbability Principle can “intertwine, braiding together and amplifying each other, to form a rope connecting events, incidents and outcomes.”

To give you a sneak preview, in Chapter 4 of the book, Professor Hand brings forward the Law of Inevitability which says “even if each of the possible outcomes has a tiny probability of occurring, it’s certain that one of them will.”  One example he gives is the stock tipster scam whereby the perpetrator sends letters to say, 1024 individuals and claims falsely to have the ability to predict the daily ups and downs of stock market.  Most of us would dismiss such a claim instinctively.  But the scammers are pretty smart people too and have their ways to combat the skepticism, making use of the law of inevitability!   The perpetrator would send 1024 distinct predictions of market movements of 10 future business days, one for each of these 1024 individuals.   If you and I are making a random guess, we would have less than 1 in 1,000 chances to get it right.  But guess what?  Since there are only 1024 possibly distinct outcomes of up-down movement over any 10 business days, one of these 1024 individuals is guaranteed to have received from the scammer the correct 10-day long pattern.  Such an incredibly impressive achievement (to the one who received the correct prediction) could be sufficient for the perpetrator to profit from the scam follows.   

In Chapter 5 of the book, Professor Hand recalls a number of interesting real stories, ranging from someone had won multiple lotteries over a short period of time to a woman saw a book in a used-book store that she used to own many years earlier.  You and I would probably react to such stories with “Wow, what an amazing coincidence!” and soon forgot about it.  Then we find ourselves react similarly soon enough with yet another such a story.  To show us why such stories seem to take place so frequently, Professor Hand introduces the Law of Truly Large Numbers that says “with a large enough number of opportunities, any outrageous thing is likely to happen.”

Take the lottery story as an example.  Most of us understand that the chance for a randomly chosen player to win a huge jackpot can be astronomically small (by design).  Further, most of us also understand that the chance of an individual would win two large jackpots of independent lotteries is exponentially smaller – the odds to win both would be a trillion to 1 if winning one is a million to 1.   But then, should we be surprised upon hearing the news that someone had won two lotteries somewhere?  It turns out that we really shouldn’t be surprised.  Part of our problem is that we were looking at the wrong measures.  While the chance of winning a large jackpot of a lottery is indeed tiny, what the news and we were actually talking about was the (expected) number of occurrence of such an event over an unspecified period of time – or how many winners would you expect to have such an unbelievable luck of winning two jackpots.   Were you ever surprised when someone did win the jackpot of a lottery?  Once you realize how many lotteries there are and how many people are playing multiple lotteries and bought many tickets regularly, why should we be surprised if once in a while, one of them won two lotteries somewhere over a long period?  The lesson is that although the probability for an event to occur is tiny, we tend to forget the number of attempts is can be much larger than we expected and consequently, the event is almost certain to happen in due time.

These two are just tips of the icebergs.   Human beings do have at least a few more blind spots that can mislead us as well.  In chapter 6, Professor Hand notes the Law of Selection that says “you can make probabilities as high as you like if you choose after the event”.  While I trust none of us would cheat by drawing the target after shooting the arrows, we do commit, from time to time, unconsciously the act of fitting questions to answers and identifying causes to result after the event has taken place.  When was the last time you heard someone attributes a good thing happened to him/her (after it happened) due to his/her prayer made earlier?  What he/she might have neglected was the other nine hundred ninety nine times when nothing memorable happened!  No one was harmed in these type of stories of course.  But what if it is a scrupulous investment advisor who is telling you examples of selected clients who had made insane profit?  Would you be so impressed and invest your life time savings to his scheme?  By the way, this is closely related to the so-called Confirmation bias that we tend to count and remember only those events that you want to include for whatever the reason.

If all these were not enough, we definitely are not helping ourselves by getting little too relaxed or sloppy.  Complementing the Look-elsewhere Effect, Professor Hand’s law of near enough says that events which are sufficiently similar are regarded as identical.  With this powerful law, you will see that we can ensure the “connection” we want to draw can be true.  

Let me show you with the following example.  Let us say you try to initiate a conversation and “connect” with a girl in a bar.  First, the Look-elsewhere effect expands the space of search – if she did not come from your town, ask if she is coming from the same county or state.   If that is not the case, the law of near enough will likely do the job – you can ask if she knows someone from the state you come from.  You will be impressed how effective this trick works.   I used to brag about I can find some connection quickly with any stranger from Taiwan – a place of 26 million people.   That was before I learned that someone has already formalized a theory of Six Degrees of Separation that everyone and everything in the world can be connected within six “connections”!   Interestingly, most of us accept matching 5 numbers is not worth much in a lottery where the jackpot requires the matching of all 6 numbers.  The trap is, while 5 deceptively close to 6, the cases of five matches is so incredibly larger than that of matching 6!

There are a couple more laws the book discusses.  You can read them at your own leisure and reach your own conclusion.  Even if you don’t agree with some of points made in the book, you should at least be aware that many legit as well as scrupulous people/organizations do use these laws to their advantage to try to get your money.  If you do find the analysis interesting and helpful.  Here is an exercise for you.  When you open a fortune cookie the next time, tell us if it is relevant and applicable to you and if any of these laws are at work.  

Sorry, the world is more boring than you and I thought.  Believe it or not, there simply aren’t as many improbable events and miracles.   Talk to you soon!

Saturday, June 9, 2012

It Is Not Hard to be Thorough


One useful trick in dissecting a complex problem and performing thorough analysis of it is what statisticians called Factorial Experiment.  Intuitively, in analyzing an phenomenon, once one identify some underlying key factors and their possible levels/values, one can generate exhaustively all possible combinations of these factors and values.   Often, with such a brute force exercise, one finds some surprises of missing cases which have not been noticed previously.

To appreciate what the techniques is, let us look at the smallest examples when there are two factors with each taking possibly two distinct levels/values.  Thus, in total, they constitute 2 by 2, or 4 cases.
Professor Daniel Kahneman has the following discussion of decision makings with what he called “the fourfold pattern” in his book Thinking, Fast and Slow.  He noted two key factors in making decisions with risks: one factor is the perceived value of the outcome which can be gains or losses.  The second factor is the likelihood of the outcome which can be high or low.  Four cases or patterns can thus be identified:
  1. High probability of gains
  2. Low probability of gains
  3. High probability of losses
  4. Low probability of losses
 Now, one can systematically analyze easily the behavior of people when they are in each of these situations.  I bet you would agree that people tend to be risk-averse (i.e., prefer certainty) when in situation 1 and 4, and be risk-seeking (i.e., prefer gambling or taking a chance) when in situation 2 and 3.    Kahneman’s examples include: being risk averse for case 1 and 4, the acceptance of unfavorable settlement when there is a 95% to win $10,000 and 5% chance to lose $10,000, respectively.   The formal is the reason why many settle a law suits even when there is a chance to win much bigger and the latter is the reason why insurance business exists and why people buy insurance even when it is overpriced.  And being risk seeking for case 2 and 3, the rejection of favorable settlements, when there is a 5% chances to win $10,000 or 95% chances to lose $10,000.  The former is the reason why gambling business exists and the latter is the reason why some people refuse to be relocated during disasters or eminent domain cases.  

I am sure you can relate or find more interesting examples and observations using such fourfold pattern or factorial design trick.  Here are two interesting ones to get you going.

Two decades back, I heard in a Broadband Networking Conference what I thought was the best characterization of the difficult notion of “virtual”.  At the early part of his keynote speech, Bob Kahn, one of "the fathers of the Internet" (along with Vint Cerf), drew on a transparency a two by two grid.  In one dimension, he noted the factor of visibility (of an object) which can be visible, or invisible. And in the other dimension, he noted the factor of existence which can be yes (is there), or no (is not there).  While we are familiar with the physical world where objects are visible and there, networking researchers have created long ago a world of virtual objects and resources which are visible but not there!  This is actually one fundamental reason why Internet-based services can be inexpensive but more volatile.   Now I think you can understand easily and intuitively what all these virtual xyz’s (world, game, reality,…) you have been hearing about are.

In closing, let me give you one more example of fourfold pattern.  In my previous life as a manager, one report of mine related the following quiz to me during our annual performance review discussion.  He asked “How would you rank order the following four types of employees?”
  1.  Smart and hardworking
  2. Stupid and hardworking?
  3. Smart and lazy?
  4. Stupid and lazy?
His insight is that stupid and lazy is preferable to stupid and hardworking since the latter produces far greater negative impact and rework to the team!  Can you guess which type of employees is he?
Talk to you soon!

Thursday, May 17, 2012

Silo Thinking


By now, you may have figured out that my last few blogs were inspired by the recent book Thinking, Fast and Slow of Professor Daniel Kahneman, a trained psychologist and a co-recipient of 2002 Nobel Prize in Economics.  The Prize citation recognized him "for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty."  

Along with many researchers and thinkers, Kaheman has been making significant contributions to the field of what is now known as Behavioral Economics that “study the effects of social, cognitive and emotional factors on the economic decisions of individuals and institutions and the consequences for market prices, returns and the resource allocation.”  

You may wonder if main stream economists did not incorporate the effects of social, cognitive and emotional factors, what was the economic decision theory like and how good are they?  In his book, Professor Kahneman recalls how he got involved in the study of decision making.  It all started one day in early 1970s when late Amos Tversky, Kahneman’s collaborator and close friend, showed him an essay that started with the sentence “The agent of economic theory is rational, selfish, and his tastes do not change.”  The statement is a concise summary of one key assumption of the dominant neoclassical economics theory and rationality in this context means “an individual acts as if balancing costs against benefits to arrive at action that maximizes personal advantage.”

What is interesting is the ensuing “what?!” moment.  As Kahneman recalls “To a psychologist, it is self-evident that people are neither fully rational nor completely selfish, and that their tastes are anything but stable.  Our two disciplines seemed to be studying different species....”  A Nobel prize-worthy work was then begun.  Indeed this story is a good illustration of why honest inter-disciplinary efforts can be very productive that bring different perspectives and thus connect otherwise isolated frames of mind.
In subsequent decades, Kahneman and other researchers have brought further insights from psychology and injected more reality into economics theory.  Behavioral economics is now a significant scientific discipline that offered more satisfactory explanations and predictions of human economic behaviors.  

One example Kahneman used in his book to illustrate the inadequacy of classical models is fairly easy to follow.  If utility of wealth is indeed the deciding factor as the classical theory postulated, then the following two games would produce similar preferences.  But intuitively, you know it cannot possibly be the case.  

1.      In addition to whatever you own, you have been given $1,000.  You are now asked to choose one of these options: 50% chance to win $1,000 OR get $500 for sure.
2.      In addition to whatever you own, you have been given $2,000.  You are now asked to choose one of these options: 50% chance to lose $1,000 OR lose $500 for sure.

With a 20/20 hindsight, you may wonder why did it take so long for experts to amend flawed models and why so many experts were oblivious to the obviously (to common men) erroneous implications when applying some methods of classical decision theory to scenarios involving losses and risks.

Economics is certainly not the only field that suffers from the ill effects of silo thinking. Researchers in all fields are often so absorbed and entrenched in their research and are so comfortable with their training, they accept the theory and rarely bother to revisit the critical assumptions.  Another distinct possibility is that some theories were never really applied in practice nor produced any serious consequences; thus there is no incentive to validate it.  Worse yet, it is far easier to make incremental enhancements to accepted theory than to establish something fundamentally new.  

In fact silo thinking is a systematic problem that does not end at scientific explorations. It is prevalent in organization and business of all kinds as well.  A vast majority of us devote most of our energy in digging inch by inch the very silo we live in that provides us a comfortable “shelter”, the same reason why most researchers don’t branch out from their familiar ground.  

Whenever one goes out of his or her own silo once in a while, good things happen.  At a minimum, it could be refreshing to see different perspectives.  Sometimes, one may even enjoy an Eureka moment like ancient Greek scholar Archimedes did or a sudden enlightenment like Zen Buddhism calls it.  As trained psychologists, Kahenman and Tversky recognized the importance of the psychological value of gains and losses in decision making that was missed by main stream economists.  Their introduction of the concept of loss aversion has since been considered the most significant contributions of psychology to behavioral economics.  Sounds pretty intuitive and easy, isn’t it?  You can make a difference too if you think out of box once in a while!

Talk to you soon!  Sign off by this frog at this well on this day.

Sunday, April 29, 2012

Our Causal Mind



Causal (not to be confused with “casual”) is a word that is not often used or heard in our daily life.  Yet the word describes a concept that is deeply imbedded in our minds and influences how you and I behave all the time.  Looking up the dictionary, you will find that the word causal means ”relating to or arising from a cause” and the noun causality (and causation) is about the cause-effect relationship of events.  That is, these words describe the familiar notion that one event (the effect) is a consequence of another event or set of events (the cause). 
 
Don’t we all resist the idea that things/events can happen with no reasons?  Don’t we all like to believe that if we had known every piece of information and actions, then we could have predicted where we would have been?  By the same token but going backward in time, don’t we also naturally feel that given an outcome, it would be possible (at least in principle) to identify what have caused it?   Unfortunately, such feelings and beliefs, while comforting, may not always be correct and sometimes are outright illusionary.  But, since it is practically impossible to replay alternate path completed with all the factors minus the causes, we happily convince ourselves to accept the universality of causation anyway even when it is not certain or true. 

Narrative fallacy (see the book The Black Swan by Nassim Taleb) is a consequence of our causal minds.  We love stories.  Journalists report and write stories of the news events.  In the process of making sense of the events and happenings, we willingly become victims of our “Illusion of Understanding”.  What makes it worse is that according to Professor Daniel Kahneman discusses in his recent book Thinking, Fast and Slow, we also have a remarkable ability in sense-making.  As we attempt to make sense of the world, we usually reconstruct a coherent story after the fact and make sure it is consistent with the outcome.  This is done regardless if one had different beliefs or predictions and psychologists call it hindsight or the outcome bias.   When talking to your aging parents, haven’t you ever wondered where did they get their distorted stories from?  Have you ever realized that one day it is where you will end up to be at too? 

What about experts’ opinions?  Consider the daily headlines of the financial market and sports news.  We are always told reasons why the stock market went up or down and why a pro sport team won or lose, after the fact.   Perhaps we can sympathize with pundits and commentators who are subject to tremendous pressure that they need to explain and predict things for which they are supposed to be the experts.  But professional stock pickers, traders, company CEOs and political leaders are not exempted from this problem either.  “Illusion of skills” and downplaying “luck” (or more accurately, chance) is necessary and natural for those who eager to justify their existence and rewards.  We the consumers of these “goods” certainly play a huge role as we provide the insatiable demand for such behaviors.

The truth is that our minds just love and yearn for causal explanations.  Statistics is a useful tool but we don’t seem to handle statistical information and complexity well.  We (well-educated ones included) routinely overlook and ignore the sound advice and logic that CORRELATION DOES NOT IMPLY CAUSATION.  Many glaring traps and examples can be found the health related areas.  When was the last time you heard a claim, a report, or controversy about what may help with an illness or improve your heath?  When was the last time you received tempting information about certain diet supplements or therapy that will make you healthy?  More often than not, such information are either interpreted incorrectly due to ignorance or simply designed to prey on our tendency of substituting causation for correlation, hoping by chance the cure will turn into certainty.

What tops all among our inadequacies is perhaps what Donald Rumsfeld, the former Secretary of Defense, referred to as “there are things we do not know we don’t know” or for short “unknown unknowns” (when he addressed the intelligence challenges early 2002, prior to the 2003 invasion of Iraq ).  This is related to the phenomenon what Professor Kahneman called WYSIATI – What You See Is All There.  It is simply impossible to take into account of information that does not even come to your mind because you never knew it is there to begin with!   One trivial consequence of this flaw is we rarely recognize or reward those tremendous efforts that prevented a disaster from happening.  Instead we celebrate heroism in a disaster all the time which is simply the mirror opposite of the former.  Sadly, out of the four quadrants including known knowns, known unknowns, unknown knowns, the quadrant of unknown unknowns is for sure the largest although we will never know how big it really is.

In sum, we misconstrue noisy data and confuse complex, non-sequential events with simple, plausible, time-ordered, causal explanations.  Satisfied with our story, we congratulate ourselves and gladly move on with confidence.  Perhaps such optimism is necessary for evolutionary reasons and survival.  In his recent book, Professor Daniel Kahneman also tells us that our brains are “wired” as two counterbalancing systems feeding and driving each other: a Fast System that is instinctive and responds instantly to received stimuli based on a model; and a Slow System which is deliberate and analytic in digesting the experiences and updating the model used by the Fast System.  The model used by our fast and intuitive system appears to have all the characteristics of a causal one.

Further, our Slow System thinks and pursues answers to the questions of whys and tries to make sense of the world “off-line”.  While it is slow and tend to be lazy (or has a limited energy?), we are lucky that as a social-economic animal, we do share the results and discoveries, and learn from each other.   The most familiar examples can be found in fields of science, engineering and technology.   We wouldn’t be where we are if it weren’t because many have devoted so much of their energy to pursue relentlessly the answers to what’s, why’s and how’s.  

Science is definitely not the only way we make sense of the world.  Spiritual and philosophical approaches are far more popular and prevalent in the history of mankind.  Confronting the difficulty of having satisfactory explanations and the impossibility of knowing and predicting accurately, religion offers a comforting solution.   To begin with, one can find his/her answers by attributing all unexplainables (and sometimes explainables as well) to some supernatural entities (gods).   Abrahamic religions for instance (Judaism, Christianity, and Islam) believe in omnipotence that god is in control of everything.  Some take a step further and believe in omnicausality that god is the cause of all things.  Indian religions (Hinduism, Buddhism included) on the other hand, discuss the concept of Karma extensively which is about the cycles of cause-effects.  All these demonstrate just how strong the desire is for a causal model of the world by human.  

Are you as surprised as I am that despite all the flaws and shortcomings, we seem to be doing ok so far?   But do you really think our causal mind is the best design?  

Talk to you soon!

Thursday, March 29, 2012

How Useful is Additional Information?



As an informed and learned man, we are exposed to large amount of data and information all the time. Some of these data are incomplete, noisy; some are confusing and contradicting, and some are misleading or downright wrong.  Over time, with some filtering and learning, we formed our own (biased) opinions, and apply our judgment to event of interest.  While the intent and process may seem innocuous, our biases could and do sometimes have unintended and/or horrific consequences.  Take the recent news from Sanford, Florida as an example.  Trayvon Martin, a 17 years old, unarmed teen was shot dead by a neighborhood watch volunteer George Zimmerman.  While many details and facts are still unavailable to public, we do know that the tragic event started when George Zimmerman, according to his account, saw the hooded teen in the neighborhood and thought “he looks suspicious”.  

In his recent book Thinking, Fast and Slow, Professor Daniel Kahneman, 2002 Nobel Laureate in Economics, discussed how our brains struggle to balance our quick intuitions and slow analytics when making judgment.   We are proud of our build-in survival instincts and life experience that helped our quick intuitive responses.  We insist, while acknowledging such responses are not always correct, they are time tested and often sound.  We are perfectly ok to have such an efficient way of cutting through complex reality to get to the guts of the resolution.  And we can’t imagine how we can survive or live without such ability.  On the flip side, words such as profiling, stereotyping, bias, prejudice are often used to describe the same intuitive conclusions.   As we live in this information age and are constantly bombarded with noisy information and news, one must wonder how good is our gut reactions and how wrong could they be.  

In the chapter “Causes trump Statistics” of the said book, Professor Kahneman gave the following example:  “A cab was involved in a hit-and-run accident at night.  Two cab companies, the Green and the Blue, operate in the city.  You are given the following data: 85% of the cabs in the city are Green and 15% are blue.  A witness identified the cab as Blue.  The court tested the reliability of the witness under the circumstances that existed on the night of the accident and concluded that the witness correctly identified each one of the two colors 80% of the time and failed 20% of the time.”  Then Professor Kahneman asked us to answer intuitively to the question “What is the probability that the cab involved in the accident was Blue rather than Green?”

I don’t know about you.  But if I have to give an answer intuitively, I would probably put my bet on the taxi involved is Blue given the seemingly pretty solid witness’ account.  In real life, each of us runs into this type of situations from time to time and offers our guess freely.  But what if the answer matters (e.g., you are a juror in such a case in court) and what if we are wrong?  

So, what is the odd that I would win my bet in absence of any further information?  In this example, we were first given the fact that only 15% of taxi in this city is Blue.   If we knew nothing else, most of us probably would agree that a reasonable guess would be there is a 15% chance that it is a Blue taxi when an accident involving a taxi occurs.   The challenge comes when we are given further and additional information since we believe we now know a lot more about the event.  

In this example, we are told that there is an eye witness who said the taxi involved is Blue.   After all, the witness is 80% reliable that seems pretty good.  In a different setup, we might be told that Blue taxi is much more prune to having accident based on historical data, etc.   The core issue remains the same however: how can we incorporate such additional information properly?

More than 300 years ago, the 18th century English mathematician Thomas Bayes addressed this very issue and developed an approach of how one could update one’s beliefs when given new evidence.   His work is fundamental to the theory of probability and statistics and bears names like Bayes’ Theorem, Bayes’ law, Bayesian Statistics, and Bayesian inference, etc.   If we applies Bayes’ Theorem to our taxi problem as Professor Kahneman suggested, we could update our estimate as follows once we are told the 2nd piece of information that there is an eye witness who identified the taxi is Blue.  

The calculation is based on the simple observation that the probability of {the taxi is Blue} and {eye witness thinks it is Blue} can be expressed in two ways with priori or posterior probabilities:  this probability is equal to the product of the posterior probability of {that taxi is Blue, given eyewitness thinks it is Blue} of interest and the probability of {eye witness thinks it is Blue}.  It can also be expressed as the product of the probability of {eyewitness thinks it is Blue, given taxi is Blue} which is 0.8 and the priori probability of {that taxi is Blue} which is 0.15.  

But the eyewitness could be wrong.  The eyewitness can be thinking that the taxi is Blue when it is actually Green and he mistaken it to be Blue (20% chance), or taxi is indeed Blue and he got it right which is 80%.  Thus the latter probability of the evidence that eyewitness think it is blue is thus 0.20x0.85 + 0.80x0.15, or 0.29.  Therefore the posterior probability of our interest that the taxi is indeed Blue given eyewitness thinks it is Blue would be the ratio of 0.80x0.15 over 0.29, or 41%.  In other words, with the additional evidence provided by the eyewitness, the chance that the taxi is blue has risen almost three folds from 15% (based on the priori statistic) to 41%.  However, the resulted probability is still smaller than the obvious reference figure of 50% from a random toss of a fair coin if you had absolutely no information other than the fact there are two colors of taxi in the city.

What the example illustrates is that the additional information (in this example, the eyewitness account) may not give us nearly as much as we thought it would.  That is, we tend to give too much significance and weight to such additional information than it really deserves.   For this example, to have the updated estimate that the taxi is Blue be better than 50% (a totally uninformed random guess), the eye witness needs to be more than 85% reliable.   Note if we want to be 95% sure, the eyewitness reliability needs to be better than 98% in this case.

You can easily create examples like this with other subjects and numbers.  Perhaps you are visiting an area and you were told in advance that the minority population there is 15%.   Perhaps you are visiting an area and you were told that a particular group of certain profile is known to be responsible for 80% of the crimes in that area.  When there is a crime reported and a witness identified the perpetrator is of minority race or fits the profile of that group.  What is the chance that crime was indeed committed by a person of minority race or of that group as eye witness claimed?

When George Zimmerman of Sanford thought Trayvon Martin looked suspicious, when he began to follow him with a hand gun in his waist, the encounter ended with a tragic shooting death of the teen.   When the law enforcement and intelligence agencies collect statistics and profile groups, when institutions collect statistics about college admission of minority or under-represented groups, shouldn’t we all be concerned about how the data are used and if there is a rush to judgment?  Shouldn’t we all be worried about making proper inferences when we pick up new knowledge? 

Talk to you soon!