Benford's Law: finding fraud and data oddities

What links fraud detection, old-fashioned log tables, and error detection in data feeds? Benford’s Law provides the link and I'll show you what it is and how you might use it.

Imagine I gave you thousands of invoices and asked you to record the first digit of the amount. Out of say, 10,000 invoices, how many would you expect to start with the number 1, how many with the number 2, and so on? Naively, you might expect 1,111 to start with a 1; 1,111 to start with a 2 and so on. But that’s not what happens in the real world. 1 occurs more often than 2, which occurs more often than 3, and so on.

The Benford’s Law story starts in 1881, when Simon Newcomb, an astronomer, was using some mathematical log tables. For those of you too young to know, these are tables of the logarithms of numbers, very useful in pre-calculator days. Newcomb noticed that the pages for logarithms beginning 1 were more well-thumbed than the other pages, indicating that people were looking for the logarithms of some numbers more than others. Being an academic, he published a paper on it.

In 1938, a physicist called Frank Benford looked at a number of datasets and found the same relationship between the first digits. For example, he looked at the first digit of addresses and found that 1 occurred more frequently than 2, which occurred more frequently than 3 and so on. He didn't just look at addresses, he looked at the first digit of physical constants, the surface area of rivers, and numbers in the Reader's Digest etc. Despite being the second person to discover this relationship, the law is named after him and not Newcomb.

It turns out, we can mathematically describe Benford’s Law as:

P(d) = log(1 + (1/d))

Where d is the numbers 1 to 9 and P(d) is the probability of the number occurring. If we plot it out we get:

This means that for some datasets we expect the first digit to be one 30.1% of the time, the second digit to be two 17.6% of the time, three to be the first digit 12.5% of the time, etc.

The why of Benford’s Law is much too complex for this blog post. It was only recently (1998) proved by Hill [Hill] and involves digging into the central limit theorem and some very fundamental statistical and probability concepts.

Going back to my accounting example, it would seem all we have to do is plot the distribution for our invoice data and compare it to Benford’s Law. If there’s a difference, then there’s fraud. But the reality is, things are more complex than that.

Benford’s Law doesn’t apply everywhere, there are some conditions:

• The data set must vary over several orders of magnitude (e.g. from 1 to 1,000)
• The data set must have dimensions, or units. For example, Euros, or mm.
• The mean is greater than the median and the skew is positive.

Collins provides a nice overview of how it can be used to detect accounting fraud [Collins]. But Linville [Linville] has poked some practical holes in its use. He conducted an experiment using graduate students to create fake test invoices (this was a research exercise, not an attempt at fraud!) that were mixed in with simulated invoice data. He found that if the fake invoices were less than 10% or so of the total dataset, the deviations from Benford’s Law were too small to be reliably detected.

Benford’s Law actually applies to all digits, not just the first. We can plot out an expected distribution for two digits as I’ve shown below. This has also been used for fraud detection as you might expect.

You can use Benford's Law to detect errors in incoming data. Let's say you have a datafeed of user addresses. You know the house numbers should obey Benford's Law, so you can work out the distribution the data actually has and compare it to the theoretical Benford's Law distribution. If the difference is above some threshold, you can set an alert. Bear in mind, it's not just addresses that follow the law, other properties of a data feed may too. A deviation from Benford"s Law doesn't tell you which particular items are wrong, but you do get a clue about which category, for example,  you might discover items starting with a 2 are too frequent. This is a special case of using the deviation of real data from an expected distribution as an error detection mechanism - a very useful data quality assurance method everyone should be using.

To truly understand Benford’s Law, you’ll need to dig deeply into statistics and possibly number theory, but using it is relatively straightforward. You should be aware it exists and know its limitations - especially if you’re looking for fraud.

References

[Collins] J. Carlton Collins, “Using Excel and Benford’s Law to detect fraud”, https://www.journalofaccountancy.com/issues/2017/apr/excel-and-benfords-law-to-detect-fraud.html
[Hill] Hill, T. P. "The First Digit Phenomenon." Amer. Sci. 86, 358-363, 1998.
[Linville] “The Problem Of False Negative Results In The Use Of Digit Analysis”, Mark Linville, The Journal of Applied Business Research, Volume 24, Number 1

Further reading

Wikipedia article https://en.wikipedia.org/wiki/Benford%27s_law
Mathworld article http://mathworld.wolfram.com/BenfordsLaw.html

How to be a compelling speaker: creating applause and tricolon

You can learn how to be a compelling speaker and, with practice, you can learn how to trigger an audience to applaud when you make your points. The techniques to grip an audience were known in antiquity and speakers have used them throughout history to great effect to further their interests. You too can learn these techniques and apply them in your talks.

In this blog post, I'm going to focus on one very powerful technique: tricolon. This is my absolute favorite of all the rhetorical devices. It’s almost magical how well it works; in fact, it’s closely related to something called an applause trigger that I’ll come to later. Tricolon has been used by almost all the major political orators of the last 50 years as we’ll see.

At its simplest, tricolon is a three-part list. Here’s an example:

Government of the people, by the people, for the people.
-Abraham Lincoln

(Image credit: WikiMedia Commons. Public domain.)

Tricolon is literally just a list of three phrases with similar grammatical and word structure. Here’s another example:

We can help all peoples to see it, to draw hope from it, and to move irresistibly toward it.
-John F Kennedy

Putting your points in a list of three seems to elevate your argument and make it much more memorable. Three-part lists are everywhere for a reason. In western cultures, three seems to have a magnetic effect – for example, we give gold, silver and bronze medals; we say snap, crackle, and pop; and we learn of life, liberty, and the pursuit of happiness. In fact, tricolon is so powerful, sometimes speakers even add ideas to lists to make three-part lists, or chop down lists of four to three.

Orators have figured out that by using three-part lists (tricolon) and managing their voice and gestures, they can sometimes trigger an audience to applaud. That’s what’s called an applause trigger. There are a handful of rhetorical techniques known to be applause triggers including tricolon, puzzle-solution, and two-part contrasts [Atkinson, Heritage]. It’s extremely hard to pull off an applause trigger and not every tricolon can work as a trigger, but it can be taught. 

A few years ago, I was at a talk on analytics and the topic turned to Microsoft PowerPoint. The speaker introduced his speech coach to make a point. The point was anodyne, but the speech coach phrased it as a tricolon, He increased the intensity of his voice to reach a crescendo on the last of his three points and the audience burst into a strong round of applause as soon as he made his last point. I knew of this technique at the time and I spotted what he was doing as he was doing it; it was really impressive to see him deliver his lines and get the applause he was looking for. Stop for a minute and think about this, a speaker got a strong round of applause by talking about PowerPoint. Do you think you could be trained to do something similar?

I know my audience for these blog posts is well-educated, intelligent, and perceptive. I’m sure you’re thinking, ‘how could something so straightforward and so simple and so easy work so well’? Because your doubt is valid, we’re going to do an exercise.

Barack Obama’s speech to the 2004 Democratic National Convention ignited his presidential campaign. In the extract below, I’ve outlined the tricolon and you can listen to the audience's response. Watch the YouTube video and read the extract below - I've structured the text to highlight the tricolon.

When we send our young men and women into harm's way, we have a solemn obligation 

1. not to fudge the numbers or shade the truth about why they're going, 
2. to care for their families while they're gone, 
3. to tend to the soldiers upon their return, 

and to never ever go to war without enough troops to 

1. win the war, 
2. secure the peace, 
3. and earn the respect of the world.

Listen to how Obama uses his voice as he goes through his tricolons. He manages to build up the audience reaction so they applaud at the end of his list. He clearly signals to the audience when it's time to applaud by his use of tricolon and his voice.

Of course, Barack Obama used this speech to propel him from senator, to candidate, to President.

My final example is, in my view, one of the finest political orators of the twentieth century. She didn’t get this good by accident, she was intensively trained and coached – which is an interesting story all by itself. The speaker is Margaret Thatcher. The occasion is her party conference in 1980. Let’s hear a brief burst of her using tricolon (1:34 on the audio on this page).

This week has demonstrated that we are a party united in 

1. purpose, 
2. strategy 
3. and resolve

And later on (27:24):

Soviet Marxism is 

1. ideologically, 
2. politically 
3. and morally bankrupt. 

If you have time, I recommend you listen to some of Margaret Thatcher's speeches regardless of whether you agree or disagree wither her politics. She was a master of rhetorical set pieces and she knew how to use tricolon and other techniques to trigger applause.

It's very hard to deliver a tricolon in a way that triggers applause, but even if you don't use it that way, it's still very much worth your while using it. It makes your arguments sound much more compelling.

In your writing and presentations, you can use tricolon to link ideas together and build an argument. For example, if I were the CEO giving a speech to rally the company, my theme might be the need to perform better than the competition.  I could just say:

The competition exists to take our market from us, we need to perform better than them.

Or I could say:

The competition exists to take our market from us, 

1. we need to out-innovate them, 
2. we need to out-deliver them, 
3. and we need to out-sell them.

Which sounds better to you?

References

[Atkinson] "Our Masters' Voices: The Language and Body-language of Politics", Max Atkinson, 1984

[Heritage et al] "Generating Applause A study of Rhetoric and Response at Party Political Conferences", http://www.sscnet.ucla.edu/soc/faculty/heritage/Site/Publications_files/APPLAUSE.pdf

Reading more

This blog post is one in a series of posts on practical rhetoric. Here's the series:

• Presentation advice from a professional comedian
• How to be more persuasive when you speak: using 'catchphrases'
• Being more persuasive in speech: anaphora

Cheating charts: the axes of evil

As you might have guessed from the title, this post is all about how you can play around with chart axes to lie like truth. It's about being evil with axes.

In the Harry Potter books, the children are taught 'Defence Against the Dark Arts' not to teach them how to be evil, but rather to teach them how to defend against evil. I'm using the same approach here; this blog post is about defending yourself against being misleading or being misled.  I'm going to show you ways that people have used chart axes to obscure the truth. But we need to be careful with blame; sometimes, charts are unintentionally deceitful, the author miscommunicated rather than set out to misinform, and sometimes it's a matter of opinion. Read what I have to say and decide for yourself.

(2x2 matrix - an example of evil axes)

Zero axis

In most cases, charts should include zero so as not to mislead about the size of an effect. Let's take house prices in London as our example. UK inflation (CPI) was 1.8% for the twelve months from January 2019 to January 2020, over the same period, London house prices increased 2.8% - not a bad increase, but we can make it look much larger.

Let's start with an honest chart.

It clearly shows a small increase, but it would be hard to get a newspaper headline from it. Imagine you were a newspaper editor and you needed to squeeze a sensationalist story from the data. You need to make the difference appear much bigger, but still have a fig leaf of decency. How can you do it? The simplest way is excluding zero and zooming in.

Imagine that we coupled it with a headline like, 'London Property Market Booms' and had an article with examples of extremely expensive houses and some anecdotes of house buying. If you just glanced at the chart and read the story, you might think the market was growing explosively. This trick works even better if you make the axes text small, reduce their contrast with the background color, or even remove them altogether.

If you're trying to be honest, most of the time, you should include zeros to truly scale the effect and not mislead. But there are exceptions. Sometimes you do want to exclude zero as in the example below.

I have some data on human body temperature over the course of a day, taken from Wikipedia. Here's a chart including zero (as in 0 centigrade).

There really doesn't seem to be much variation does there? It looks like the human body temperature stays more or less constant during the day. In fact, the data looks just like noise. I could flatten the chart further by using degrees Kelvin or even showing a Fahrenheit scale starting from zero. 

When we zoom in and exclude zero, a clearer picture emerges.

Plainly, human body temperature does change during the day. Given the fact that a few degrees difference in body temperature can make the difference between someone who's fine and someone who's in medical danger, the second chart is a better and more honest and useful representation.

If you want to cheat and misrepresent, here's what you should do:

• If you want to exaggerate a small difference, don't include zero and zoom into your chart to expand the difference.
• If you want to suppress a difference, include zero and choose units that minimize the difference.

If you want to be honest:

• Include zero by default.
• Don't include zero when you're looking at small changes and the changes matter, in this case, exclude zero to focus on the change.

Extending the axis

This is a really fun way to mislead people and it's something I've only seen recently. You can extend the perception of the axis to reduce the effect. Let's use the same election example I used in my blog post on pie (lie) charts. Imagine there are four parties standing in an election and you have a record of what percentage of the vote each candidate and party received. Here's an honest bar chart showing the results.

Plainly, the Bird party did very badly (15% of the vote). Now let's see if we can minimize the scale of their defeat by redrawing the chart in a deceitful way. Let's remove the x-axis, extend the y-axis labels, color and box the labels, and introduce some bar coloring.

It's still obviously a defeat, but we've made it look much smaller. If you take the time to look, it's obvious that something funky is going on here, but most people don't have time and don't look closely.

If you want to be honest, don't play around with axis labels and colors.

Unequal steps

If you want to imply things are getting worse, or better, when they're not, then a good option is to use unequal axis scaling. Most viewers expect that an axis will scale consistently, for example, an axis might be labeled 1, 2, 3, 4, 5, 6, 7, 8, 9, 10  and more sophisticated viewers might be very comfortable with log scales, for example, axis labels 1,10, 100, 1000. Almost no one can interpret what unequal scaling means, which makes it great for evil. To make your deception even better, use a line chart (which implies continuity) rather than a bar chart (which implies category).

Let's take an example that appeared in the media, US gas prices in 2012. The AAA produces a daily set of gas price data. This has today's price, yesterday's price, last week's price, last month's price, and last year's price. It's not the greatest presentation of data and it's hard to pick out trends, but at least the data exists - and more importantly, they don't chart it. In 2012, a US media outlet (who shall remain nameless) took the data and ran a story on gas price increases under Obama. Here's my version of their chart.

At a quick glance, it looks like there was a massive increase. But was there? The periods on the x-axis aren't equal and they've used a line to indicate a continuous variable. The AAA data quotes last month's number, but that isn't shown here, why? The y axis starts at $2.80 which is an odd choice, more rational choices might have been $3.00 or $0. If you take the time to look at the chart, it's really hard to draw any conclusions, but most people don't have the time and will just conclude 'gas prices up under Obama'.

If you really want to mislead, use unequal scaling and a line chart.

Scale inversion

If you really, really want to mislead, choose a scale inversion. 

I'm going to show you one of the most controversial charts of the last ten years. The author has vigorously defended their work, and after reading their comments, I understand that they had no intention to deceive. Because I don't wish to make the author's life more difficult, I'm not going to name them or give you their employer's name.

The chart below shows homicides in Florida and what happened when the 'Stand Your Ground' Law was enacted. Before reading on, how would you interpret the chart?

Almost everyone I've spoken to interprets the chart as implying that homicides went down. But look at the y axis. It's inverted. Here's how the plot would look if the author had chosen normal scaling.

This conveys a hugely different message.

The author wasn't trying to mislead here, rather they were trying to use art to make a more emotionally informative representation of the data. You can judge for yourself whether they succeeded or not. This raises the more general topic of who is visualizing data and how it's done. 

In the last few years, there's been a tremendous rise in the use of infographics for all kinds of topics. These tend to be more poster art than information sharing, which leads us to a problem. In the information world, a large number of informal practices have grown up around how to display data in a truthful way. Infographics are sometimes created by people familiar with these practices, but sometimes not. When designers start using artist interpretation to make data more impactful, we can get distortions and unintentionally misleading people. Personally, I think infographics are little more than visual fluff.

Getting back to where I started in this section, scale inversion is a wonderful way of reversing the evidence.

Log plots

This isn't so much deceit as obfuscation or confusion.  

A logarithmic scale is one that varies logarithmically, so instead of an axis increasing like 1,2,3,4,5, it increases like 1, 10, 100, 1000, 10000. Logarithmic scales are used when data varies by orders of magnitude. 

Unfortunately, many viewers aren't familiar with the idea and it can be hard to interpret, a good example being the recent coronavirus chart in a New York Times article. Here's the chart:

(Imaged credit: New York Times, copyright New York Times)

The logarithmic axis is the y axis. What conclusions would you draw about the coronavirus from this chart? I've used log plots for years and I struggled to understand what this chart means. 

2x2 charts

2x2 charts are a special case of confusion with axis. Unfortunately, they're beloved of MBA courses and books on management and marketing. Let's take the classic BCG product matrix as an example. In the 1960s, the consulting company BCG came up with a way for companies to view their product portfolio and make more rational product investment decisions. They recommended plotting market share on the x-axis, growth on the y axis, and dividing the plot into four quadrants, each with a name, you can read more about it here. Here's a representation of their matrix.

Note that although the axes are marked, there's no scale and it's not clear where the quadrant lines are drawn. In practice, companies using this methodology may well draw scales, but in almost all cases you find on the internet, there are no scales.

The BCG matrix is just one of a large number of 2x2 matrices you can find out there. Very few of them have any kind of scale, so it's very hard to understand and interpret what they mean in practice. Bear in mind that they often imply quite different management choices for different chart quadrants, but who's in what quadrant may depend on exactly where the quadrant boundaries are drawn, and that's almost never made clear. It's really tempting to say that you need to employ consultants to tell you what they mean and to interpret the charts for you.

I'm not a fan of 2x2 matrices because I find that they confuse rather than enlighten, but if you want to produce a chart that looks pretty and requires you to interpret it for your management, a 2x2 matrix might well be the place to go.

You can fool all the people some of the time and some of the people all the time

If you know what you're looking for, you can see through deceit or malpractice with some effort. But if you're in a hurry, not paying attention, or a chart is flashed on the screen for a short period of time, a chart with evil axes will probably slip by your defenses against the dark arts.

In many ways, playing around with chart axes is one of the easiest ways to mislead people. I've shown you how people have been evil with axes in the hope that you'll be truthful and honest in your own visualizations.

I'd love to hear what you think about the 'axes of evil'. Have you come across other axis manipulations that I haven't included here?

Pie charts are lie charts

There are lots of chart types, but if you want to lie or mislead people, the best chart to use is the pie chart. I’m going to show you how to distort reality with pie charts, not so you can be a liar, but so you know never to use pie charts and to choose more honest visualizations.

Let's start with the one positive thing I know about pie charts: they're called camembert charts in France and cake charts in Germany. On balance, I prefer the French term, but we're probably stuck with the English term. Unlike camembert, pie charts often leave a bad taste in my mouth and I'll show you why.

(Camembert cheese - image credit: Coyau, Wikipedia - license : Creative Commons)

Take a look at the pie chart below. Can you put the six slices in order from largest to smallest? What percentages do you think the slices represent?

Here’s how I’ve misled you:

• Offset the slices from the 12 o’clock position to make size comparison harder. I've robbed you of the convenient 'clock face' frame of reference.
• Not put the slices in order (largest to smallest). Humans are bad at judging the relative sizes of areas and by playing with the order, I'm making it even harder.
• Not labeled the slices. This ought to be standard practice, but shockingly often isn't.

The actual percentages are:

Gray	20.9
Green	17.5
Light blue	16.8
Dark blue	16.1
Yellow	15.4
Orange	13.3

How close were you? How good was my attempt to deceive you?

Let’s use a bar chart to represent the same data.

Simple, clear, unambiguous.

I've read guidance that suggests you should only use a pie chart if you're showing two quantities that are obviously unequal. This gives the so-called pac-man pie charts. Even here, I think there are better representations, and our old-friend the bar chart would work better (albeit less interestingly).

Now let’s look at the king of deceptive practices, the 3d pie chart. This one is great because you can thoroughly mislead while still claiming to be honest. I’m going to work through a short deceptive example.

Let’s imagine there are four political parties standing in an election. The percentage results are below.

Dog	36
Cat	28
Mouse	21
Bird	15

You work for Bird, which unfortunately got the lowest share of the vote. Your job is to deceive the electorate into thinking Bird did much better than they did.

You can obscure the result by showing it as a pie chart without number labels. You can even mute the opposition colors to fool the eye. But you can go one better. You can create a 3d pie chart with shifted perspective and 'point explosion' using the data I gave above like so.

Here's what I did to create the chart:

• Took the data above as my starting point and created a pie chart.
• Rotated the chart so my slice was at the bottom.
• Made the pie chart 3d.
• Changed the perspective to emphasize my party.
• Used 'point explosion' to pull my slice out of the main body of the chart to emphasize it.
• Used shading.

This now makes it look like Bird was a serious contender in the election. The fraction of the chart area taken up with the Bird party’s color is completely disproportionate to their voter share. But you can claim honesty because the slice is still the correct proportion if the chart was viewed from above. If challenged, you can turn it into a technical/academic debate about data visualization that will turn off most people and make your opponents sound like they’re nit-picking.

You don’t have to go this far to mislead with a pie chart. All you have to do is increase the cognitive burden to interpret a chart. Some, maybe even all, of your audience might not spot what you’re trying to hide because they’re in a hurry. You can mislead some of your audience all of the time.

I want to be clear, I'm telling you about these deceptive practices so you can avoid them. There are good reasons why honest analysts don’t use pie charts. In fact, I would go one stage further; if you see a pie chart, be on your guard against dishonesty. As one of my colleagues used to say, ‘friends don’t let friends use pie charts’.

What I learned from a day in the woods

Leadership in the woods

A few years ago, I went on a one-day leadership course outdoors. We did various team-building and leadership activities, all based on outdoor exercises. I gained something from the experience, but there were positives and negatives. I would send people on a similar course, but with reservations. Here’s what happened, what I got out of it, and what I would do differently.

(Image credit: Mike Woodward, copyright Mike Woodward)

What happened

The group of us that did this course were all employees of the same company, company X, but from different departments. Some of us worked together occasionally, others did not. We knew one another, just not very well. The goal of the course was to prepare us for leadership positions.

The course took place at a facility in the countryside, not too far from civilization. This definitely wasn't an extreme survival course; the worst survival hardship was getting the wrong sandwich at lunchtime.

Once we'd all arrived, we were briefed on the day, split into groups, and given exercises to do on outdoor equipment. Before each exercise, we were informed of its purpose, and instructors helped us through it. 

One exercise was walking across a log suspended about 5m in the air; all perfectly safe because we wore harnesses and helmets etc. The instructor said the point of the exercise was to face fear and uncertainty and move forward with the support of the team. The team was encouraged to shout positive things to the person walking across; but nothing that might cause them to fall! However, the main instructor left halfway through, leaving us with more junior staff who focused on safety and didn’t reinforce the message about teamwork and support. 

In another exercise, we were led around blindfolded to build trust. 

The rest of the day went on in a similar vein, with similar exercises, and we had a debriefing session at the end of it.

What happened afterwards

I liked the program a lot and I took the message to heart about teamwork and support. Some months later, I faced a business decision that made me very nervous. I thought about my experience walking the log 5m in the air, and I went forward with my decision with more confidence (if I can do that, I can do this). However, my log walking boost wore off after about a year. Overall, the gains I made from the course only lasted for about twelve months.

Smaller, but not insignificant benefits of the whole thing for me were a day out of the office and the sense that my employer was investing in me.

There was a major problem with the day though; some people absolutely hated it. They hated the exercises, they hated having people watch them fail, and they hated the whole idea of running around in the woods; they got nothing out of the day. Bear in mind, this kind of group physical activity may bring back painful school memories for some people, and this is what happened when I was there. There was just too much baggage to learn. Although I felt good about the course, some people felt worse about the company for making them go and I'm sympathetic to their position. Did this make them less suitable as managers? I don’t think so.

Lessons learned

Would I spend corporate money to run an event like this again? Yes, but with reservations.

• I would consider very carefully people’s objections to these kinds of events. I would not coerce anyone to go. If there were a number of people firmly opposed, I would try and find something else. 
• I would consider disabled access very carefully. If someone on the team was a wheelchair user, for example, I would execute the program in an inclusive way, if at all.
• I would make sure the teamwork and leadership message was reinforced all the time. There would be a briefing before the day, and afterward. 
• I would schedule something like this once every year or two to reinforce the learning.

There are things to learn from a day in the woods, but maybe not everyone learns the same things.

The Monty Hall Problem

Everyone thinks they understand probability, but every so often, something comes along that shows that maybe you don’t actually understand it at all. The Monty Hall problem is a great example of something that seems very counterintuitive and teaches us to be very wary of "common sense".

The problem got its fame from a 1990 column written by Marilyn vos Savant in Parade magazine. She posed the problem and provided the solution, but the solution seemed so counterintuitive that several math professors and many PhDs wrote to her saying she was incorrect. The discussion was so intense, it even reached the pages of the New York Times. But vos Savant was indeed correct.

(Monty Hall left (1976) - image credit: ABC Television - source Wikimedia Commons, no known copyright, Marilyn vos Savant right (2017) - image credit: Nathan Hill via Wikimedia Commons - Creative Commons License.  Note: the reason why the photos are from different years/ages is the availability of open-source images.)

The problem is loosely based on a real person and a real quiz show. In the US, there’s a long-running quiz show called ‘Let’s make a deal’, and its host for many years was Monty Hall, in whose honor the problem is named. Monty Hall was aware of the fame of the problem and had some interesting things to say about it.

Vos Savant posed the Monty Hall problem in this form:

• A quiz show host shows a contestant three doors. Behind two of them is a goat and behind one of them is a car. The goal is to win the car.
• The host asked the contestant to choose a door, but not open it.
• Once the contestant has chosen a door, the host opens one of the other doors and shows the contestant a goat. The contestant now knows that there’s a goat behind that door, but he or she doesn’t know which of the other two doors the car’s behind.
• Here’s the key question: the host asks the contestant "do you want to change doors?".
• Once the contestant decided whether to switch or not, the host opens the contestant's chosen door and the contestant wins the car or a goat.
• Should the contestant change doors when asked by the host? Why?

What do you think the probability of winning is if the contestant does not change doors? What do you think the probability of winning is if they do?

Here are the results.

• If the contestant sticks with their choice, they have a ⅓ chance of winning.
• If the contestant changes doors, they have a ⅔ chance of winning.

What?

This is probably not what you expected, so let’s investigate what’s going on.

I’m going to start with a really simple version of the game. The host shows me three doors and asks me to choose one. There’s a ⅓ probability of the car being behind my door and ⅔ probability of the car being behind the other two doors.

Now, let’s add in the host opening one of the other doors I haven’t chosen, showing me a goat, and asking me if I want to change doors. If I don’t change doors, the probability of me winning is ⅓ because I haven’t taken into account the extra information the host has given me.

What happens if I change my strategy? When I made my initial choice of doors, there was a ⅔ probability the car was behind one of the other two doors. That can't change. Whatever happens, there are still three doors and the car must be behind one of them. There’s a ⅔ probability that the car is behind one of the two doors.

Here’s where the magic happens. When the host opens a door and shows me a goat, there’s now a 0 probability that the car’s behind that door. But there was a ⅔ probability the car was behind one of the two doors before, so this must mean there’s a ⅔ probability the car is behind the remaining door!

There are more formal proofs of the correctness of this solution, but I won’t go into them here. For those of you into Bayes theorem, there’s a really nice formal proof.

I know some of you are probably completely unconvinced. I was at first too. Years ago, I wrote a simulator and did 1,000,000 simulations of the game. Guess what? Sticking gave a ⅓ probability and changing gave a ⅔ probability. You don’t even have to write a simulator anymore, there are many websites offering simulations of the game so you can try different strategies.

If you want to investigate the problem in-depth, read Rosenhouse's book. It's 174 pages on this problem alone, covering the media furor, basic probability theory, Bayes theory, and various variations of the game. It pretty much beats the problem to death.

The Monty Hall problem is a fun problem, but it does serve to illustrate a more serious point. Probability theory is often much more complex than it first appears and the truth can be counter-intuitive. The problem teaches us humility. If you’re making business decisions on multiple probabilities, are you sure you’ve correctly worked out the odds?

References

• The Wikipedia article on the Monty Hall problem is a great place to start.
• New York Times article about the 1990 furor with some background on the problem.
• Washington Post article on the problem.
• 'The Monty Hall Problem', Jason Rosenhouse - is an entire book on various aspects of the problem. It's 174 pages long but still doesn't go into some aspects of it (e.g. the quantum variation).

Urban myths are poor motivators

Getting people to work harder by lying to them

I like motivational stories. Hearing about how people overcame adversity and achieved success or redemption can be inspiring. But there can be a problem with using stories as motivators; some of them aren't true. I'm going to look at one such motivational story that's common on the internet, describe its old and modern forms, and take it to pieces. Let's start with the original version of the story.

The original fake story - Christopher Wren and the bricklayers

(Sir Christopher Wren. Image credit: Wellcome Images Wikimedia Commons, Creative Commons License)

Sir Christopher Wren was one of the greatest English architects. He was commissioned to design a replacement for St Paul's Cathedral which was burned to the ground in the devastating 1666 Great Fire of London. So far, all of this is well-established history.

The story goes that Sir Christopher was inspecting the construction work one day when he met three bricklayers. He asked them what they were doing.

The first bricklayer said, "I'm laying bricks. I'm doing it to feed my family."

The second bricklayer said, "I'm a builder. I'm doing it because I'm proud of my work."

The third bricklayer said, with a gleam in his eye, "I'm building a cathedral that will last a thousand years and be a wonder for the ages".

Some versions of the story stop here, other versions make the third bricklayer a future manager, or the most productive, or give him some other desirable property.

The story is meant to inspire people to see the bigger picture and feel motivated about being something larger than themselves. Plainly, the listener is expected to identify with the third bricklayer. But there are two problems with the story: internal and external.

Children's stories are for children

In many versions of the story, it doesn't say who was the better bricklayer. Even if the third bricklayer was the best, or a future manager, or some other good thing, was this because of his vision, or was it a coincidence? Was the third bricklayer being inspirational or was he trying to curry favor with Sir Christopher?

It seems astonishingly unlikely that in 1671, on the basis of a single conversation, anyone would record bricklayer productivity or future career trajectory. Maybe Sir Christopher was so motivated by the third bricklayer that he did both.

The most important problem with this story is the veracity. I couldn't find this story in any biography of Sir Christopher Wren or any academic writing about his life. With some internet sleuthing, I found what appears to be the first occurrence of this story in a 1927 religious inspirational book (''What can a man believe?" [Barton]). The book gives no reference for the story's provenance.

To put it bluntly: this story was probably made up and doesn't stand up to any scrutiny.

A made-up story about a janitor

There's a more modern version of this story, this time set in the 1960s. President Kennedy was visiting a NASA establishment where he saw a janitor sweeping the floor. The President asked the janitor what he was doing and the janitor said "I'm helping put a man on the moon". Once again, there's no evidence that this ever happened.

The odd thing about the moon landing story is there are very well-documented examples of NASA staff commenting on how motivated they felt [Wiseman]. The flip side is, there's the well-known fact that many were so motivated to work long hours to achieve the moon landing that their marriages ended or they turned to alcohol [Rose]. Leaving aside the negative effects, it's easy to find verifiable quotes that tell the same story as the fake janitor story, so why use something untrue when the truth doesn't take much more effort?

'I can motivate people by telling them fairy stories'

Most of the power of motivational stories relies on their basis in truth. If I told you stories to motivate you and then admitted that they probably weren't true, do you think my coaching would be successful? What if I told you a motivational story that I told you was true, but you later found out was made up, would it undermine my credibility?

There are great and true stories of success, redemption, leadership, and sacrifice that you can use to inspire yourself and others. I'm in favor of using true stories, but I'm against using made-up stories because it undermines my leadership, and frankly, it's an insult to the intelligence of my team.

References

[Barton] "What can a man believe?", Bruce Barton, 1927
[Rose] https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/RoseRG/RoseRG_11-8-99.htm
[Wiseman] "Moonshot: What Landing a Man on the Moon Teaches Us About Collaboration, Creativity, and the Mind-set for Success", Richard Wiseman