Subscribe To Our Newsletter

Get tips and tools to tell your data story better.

No, thanks

 In Articles, Current Events, Data Analysis Concepts Simplified, Experts

Judea Pearl is a giant in the world of artificial intelligence. His work contributed significantly to today’s machines’ ability to play chess or drive cars. But he’s not impressed. Pearl feels our artificially intelligent creations are lacking something: the ability to use causal reasoning.

Pearl recently gave an interview to Quanta Magazine, in which he spoke about his hopes for AI’s next step. (He has also outlined those hopes more completely in his book, The Book of Why: The New Science of Cause and Effect.) His understanding of — and his insight into — the world of machine learning is fascinating, and his thoughts on how causality fits in feel relevant to a lot of things I talk about here.


How Does AI Work Now?

“Three decades ago, a prime challenge in artificial intelligence research was to program machines to associate a potential cause to a set of observable conditions. Pearl figured out how to do that using a scheme called Bayesian networks.”

I’ve talked about Bayesian methods of data analysis before. In contrast to traditional analysis, Bayesian data analysis allows us to incorporate information we know about similar data (the “prior”) before we begin to calculate results. This “reasoning by association,” as Pearl calls it, was a big step for machines initially. But reliance on it has left us stuck in the land of correlation — not causation.


How is Causal Reasoning Different?

“All the impressive achievements of deep learning amount to just curve fitting,” Pearl says.

The Bayesian methods used in machine learning let artificial intelligence associate different data points, factors, or events without probing whether or not there is a causal relationship. What does that mean? In the words of every data scientist ever:

Correlation is not causation.

Pearl’s goal is for machines to stop blindly connecting things based on patterns in data and start looking for causation. Rather than just recognizing that two points of data are connected, he wants AI to be able to ask why one point affects another. He points out that a lack of causal reasoning is the roadblock to computers with near-human intelligence and communication skills. (A theory that is not overly popular among his colleagues.)


What Does This Mean for the Future?

We’ve gotten very good at building machines that can crunch vast quantities of data — and find relationships in that data. What we haven’t yet achieved is teaching those machines to understand data relationships.

We can use AI to develop predictions for a lot of different things. But we can’t use it to determine causality. Understanding this is crucial as we march towards a more data-driven society. Until we make machines smarter — or maybe, more human? — we need to remember their shortcomings.

Want to know more about machine learning and causal reasoning? Need help determining if your data shows correlation or causation? Our goal at Datassist is to help journalists, nonprofits, and social sector organizations harness the power of data. We can teach, advise, and assist you so you can tell a compelling, honest data story. Reach out today to discuss your needs.

Recommended Posts

Start typing and press Enter to search

The New York Times’ new live polling project is amazing — and educational.Dataset search is a valuable new tool - but it’s not without limitations.