Subscribe To Our Newsletter

Get tips and tools to tell your data story better.

No, thanks

 In Data Analysis Concepts Simplified, Data Analysis Tools, Data Resources for Nonprofits, Experts, Team

Tennis legend Roger Federer holds a number of enviable records in his sport, but also one that his peers are likely less anxious to take away from him. Thanks to Simpson’s paradox, Federer holds the worst record for matches where the loser of the match garnered more points than the winner.

Huh?

The explanation is one part nuances of tennis scoring and one part statistical phenomenon. As a lead-up to the course I’ll be leading in September on Crafting Data Stories, I’m going to cover some of the basics of statistics here over the next few weeks. If you’re still trying to work out how a star like Federer ended up with such a dubious-sounding record, read on for our first lesson in this series: Understanding Simpson’s paradox.

When the Loser Outscores the Winner

Federer fans out there are probably grumbling at the fact that I’ve saddled your idol with such a terrible title, so let me explain. First of all, experts believe that this record actually proves Federer is a superior player. Second, here’s how it works.

Simpson’s paradox occurs when players win more individual points than their opponent but still lose the match. For example:

  • First game score: 0-6
  • Second game score: 7-5
  • Third game score: 7-5

So Player A scored a total of 14 points, while Player B scored a total of 16. But tennis matches are decided not by total points. Player A won two matches to Player B’s single victory, so Player A wins the match, despite Player B’s higher score.

This is super weird, but it’s a really common statistical paradox — one you should make sure you understand if you’re going to work with data.

Girls Gone Average, Averages Gone Wild

Back in the 70’s, the University of California at Berkeley was accused of sexual discrimination when someone noticed the school’s graduate programs accepted 44% of male applicants and only 35% of female applicants. Sounds pretty cut and dried, right?

Simpson’s paradox can lead us to incorrect conclusions if we’re not careful.

But it wasn’t.

When applications were broken down by department, the numbers told a different story. Men applied more often to programs in the sciences, which require specific skills but accept a large percentage of qualified applicants. In contrast, more women applied to humanities programs, which had fewer slots — and therefore, accepted a smaller percentage applicants.

When the data was pooled properly, there was actually a small bias in favour of women — not against them. Vudlab has created a number of fantastic interactive visualizations that illustrate Simpson’s paradox in this instance more clearly.

Breaking Down Simpson’s Paradox

Let’s say two people — we’ll call them Homer and Marge — are taking exams over the course of two weeks.

  • Week 1: Homer only takes one exam and fails it; Marge takes four exams and passes one of them
  • Week 2: Homer takes four exams and passes three of them; Marge takes her final exam and passes it

Marge scored a higher percentage than Homer both weeks.
Even though Marge performed better than Homer in both weeks, Homer passed more of his exams than Marge did overall. How does that even work?

The percentages are misleading because the sample sizes are different.Both weeks, Marge did better on a higher percentage of her tests than Homer. The problem lies in the sample size: Marge and Homer didn’t take the same number of exams either week.

When we look at the combined totals for each week, the sample sizes are equal and provide a more accurate picture of overall performance. That’s where Simpson’s paradox comes in.

When a trend appears in different data groups but seems to disappear — or even reverse — when the groups are combined, we call this Simpson’s paradox. Whether you’re comparing tennis scores, university admissions, or exam pass rates, it’s important to be sure you understand what your data is really saying. Understanding data relationships and ensuring you have enough context for the data you’re examining are key ways to avoid being tripped up by Simpson’s paradox.

Avoid the Trap of Simpson’s Paradox

Want to learn more about using data to support your work as part of a nonprofit or social sector organization? You’re in luck! You can sign up today for the Knight Center course I’m leading on Crafting Data Stories. (This course will be a little different than the MOOC I ran earlier this year with Alberto Cairo, so make sure you check it out early to avoid disappointment.)

Don’t have the time to take a statistics course? Need some expert, hands-on help with your data? The team at Datassist is at your service. Get in touch with us now to discuss your project.

Recommended Posts

Start typing and press Enter to search

Don’t overlook the importance of telling compelling data stories.How well do you really understand the statistical terms in the research you’re using?