Yesterday, Greg Satell wrote in Forbes about the importance of combining intuition with big data. Satell focuses on the difference in how “real” gut feelings and our own personal experience seem to most of us, contrasted with data and algorithms. Satell was speaking specifically to marketers to caution against going with intuition. As I have previously discussed, this difference in intuition and data-driven decisions matters across the board, in business, marketing, journalism, government agencies, and non-profit environments.
In It’s Not Personal – Evidence-Based Decisionmaking, I agree with the point Satell is making by saying that, in order to reap the benefits of big data for real social change and organizational growth, organizations must develop the culture and skill set that allows them to become data driven. They must be willing to move away from gut feelings and intuition and move towards evidence. While it may be indispensable at times to bring in an expert data scientist to advise, being data-informed and data-literate must become part of the organization’s DNA. The secret is not to collect more data but smarter data – data that supports decision-making.
Satell says that we “should be skeptical of the confidence and passion”, as opposed to the consistent and accurate information that data provides, emphasizing that imagination and the creative comes at the front end, “Algorithms are great at optimization, but terrible at imagination.”
I submit that, though a data driven culture and evidence-based decisions are critical, the creative passion and personal judgment are also needed throughout the process. Successfully exploiting the value in big data requires experimentation and exploration. As the big data evolution moves from data “analysis” to “intelligence”, successful firms are finding that, in the hands of an experienced data scientist, data can effectively:
– Tell stories and communicate vivid, memorable results.
– Provide relevant information into evidence-based decision processes.
– Integrate data analysis, modeling, and predictions into real-time processes.
A week ago, Mike Rowe, of the “Dirty Jobs” television series, summed up the relationship between passion or gut feelings and gathering evidence, “I would never advise anyone to “follow their passion” until I understand who they are, what they want, and why they want it. Even then, I’d be cautious. Passion is too important to be without, but too fickle to be guided by. Which is why I’m more inclined to say, “Don’t Follow Your Passion, But Always Bring it With You.””
In From Data “Analytics” to “Intelligence” – The need for experienced judgment, I showed how, even without extensive data, numbers can give us insights, and that intuitive understanding of the organization and objectives must be behind the data-gathering and analysis. We can use data to live healthier, more productive, more enjoyable lives. However, effectively using data is both an art and a science. “At this very moment, there’s an odds-on chance that someone in your organization is making a poor decision on the basis of information that was enormously expensive to collect … Investments in analytics can be useless, even harmful, unless employees can incorporate that data into complex decision making,” wrote Shah, Home, and Capella in HBR Magazine.
Michael Flowers, chief analytics officer for New York City under former Mayor Michael Bloomberg, said in the WSJ CIO, “To ask the right questions, analysts and decision makers alike need a deep intuitive understanding of the organization, its strategy, and its objectives. He explains, “I think expert intuition is the major missing component of all the chatter out there about analytics and being data driven.”
This integration of intuition and analytics is summed up well in the video, The Power of Data and What It Can Tell Us, “Whether creating new products or looking for ways to gain competitive advantage, the job calls for curiosity and an entrepreneurial outlook.”
My passion is visual communication of complex data in a way that the data you are reporting about your cause moves people to act. Datassist provides reporting that empowers your organization and partners to clearly understand the path you are going down together to make a difference. With the new launch of Datassist’s online presence you can join me in this movement by signing up for our monthly Resource List.