I recently had the honour of being a guest blogger on the American Evaluation Association’s 365 blog. For those of you who don’t get over to the AEA site, I’m sharing the post here as well.
Data is not objective. Full stop. There is the potential for bias in every stage of your data’s life cycle, and if you’re not careful, you can end up hurting the very cause you’re trying to help.
I’ve been doing data analysis around the globe for a long time. I’m an advocate for basing all our work on ethics and equity. My presentation on Feminist Data Analysis focuses on step-by-step processes to eliminate sexism, racism, homophobia, and more from data. This strategy requires us to examine the assumptions embedded in our habitual data practices, including:
- Where power dynamics are coming into play
- What assumptions and values are being prioritized
- Who is benefiting from all aspects of our data and analysis decisions
I like to think about projects in terms of a seven-step data life cycle. The worldviews and hidden, implicit biases of people involved are embedded in every single stage of data and evaluation. And each step presents us with the opportunity to increase equity, inclusion, and fairness.
Examining Your Data’s Life Cycle
In my talk, I walk the audience through the steps of the data life cycle and provide key questions and tools that help identify and correct bias at each step. (The full presentation, including lists of questions to ask at each step, is available for anyone to see here.)
For example, in the Project Design Step, constructing the methodology of any data project comes with many potential equity pitfalls. Probably the most prevalent bias here is toward comfort. What do the people involved know how to do?
We rely on a staggering number of accidentally sexist or racist methods — and it’s often due to the limits of understanding, training, or comfort level. The design of a project is inherently subjective because it’s constrained by the limits of what the people running it think to measure. Big donors also tend to direct their funds toward what’s comfortable (and often monolithic). You can almost forgive someone who always runs RCTs to try to answer all questions. Almost, but not quite.
We often view the Data Analysis step as the most objective and free from bias. But in reality, there are a huge number of assumptions, interpretations, and conceptual biases that are an inextricable part of data analysis. You can, for example, easily build a statistical impact evaluation model that is technically correct and simultaneously biased against women (or other vulnerable groups).
Incorporating Equity into Your Data Life Cycle
More and more people are understanding the importance of incorporating equity into every stage of the data life cycle. Catherine D’Ignazio and Lauren Klein (both of whom we love here at Datassist) have a draft online of their new book on feminist data.
Many of the participants in the event where I first gave this presentation shared their own experiences, suggested tools and resources, and stood in long lines afterward to ask questions. The conversations I had with some of them led to the development of We All Count, a project for equity in data. Join us there as we learn together and share tools, tips, and stories.
And of course, the team at Datassist is always here to help. If you’re struggling with inequity in your data, we can help. Get in touch now.