Our Data Exploration and Storytelling course is drawing to a close, and I’d like to thank all of the 6,000+ students — from 140 countries! — who joined us to learn, discuss and share. We’re already looking forward to running another course like this — Alberto and I have so enjoyed working with you all and seeing the unique perspectives from which you’re approaching your data.
During this final week or so of the course, we’ve been approached by many students who want to know what we feel are the most important takeaways when studying statistics. To answer that question properly, I’m going to put the series we just started about Nonprofits and RCTs on hold and dedicate my next two blog posts to the things I think are most important for our data journalism students to hold onto. This week’s topic: the ecological fallacy.
I posted not that long ago that data journalists need to understand the concept of ecological fallacy — or they need to stop using data. But I posted it before the course began, and I can’t overstate how important understanding this concept is — so I’m going to emphasize it some more.
Are You Asking the Right Questions?
The ecological fallacy is probably one of the most common — and most serious — errors I see in data journalism. It crops up time and time again, and what’s particularly treacherous about it is that, unlike cases where data is deliberately manipulated to support a specific argument, when the ecological fallacy occurs, it is almost always done by someone with the best of intentions who simply lacks a basic understanding of what their data is saying.
Your data won’t always be able to answer the question you wanted it to answer — and the question it will answer may not be as sexy as you hoped.
It’s important to remember that data analysis is not a two-way street. While you can use data about individuals you’ve studied to make generalizations about the group as a whole, you cannot apply that principle in the other direction — using data about a group to make statements about individual members of that group is what ecological fallacy is all about.
What Does Ecological Fallacy Look Like?
Examples — both hypothetical and real-world instances — can be found all over the web:
- Newspapers making assumptions about individual women’s choices based on declining abortion rates
- The faulty conclusion that all students in the class with the highest national score in mathematics must be exceptional at math
- Reports incorrectly asserting that an increase in antidepressant prescriptions nationally indicates a crisis of depression among citizens
- The persistent stereotype that all women are bad drivers
I prefer to think of ecological fallacy in terms of frogs, lily pads and ponds:
The easiest way to check if you’re guilty of ecological fallacy is to check if you’re telling a story that goes up the environment you’re studying or one that goes down. Data on individuals can be aggregated to tell a story about the group as a whole; data about groups can not be dissected to tell stories about individuals in that group.
Treat Your Data Right
At Datassist, our sole goal is to help data journalists, government agencies and nonprofits tell the best story they can with their data. If you’re struggling with data collection, analysis, or visualization, we can help. Let us help you use data to make a difference — check out our portfolio or get in touch with us today.