Ah, January! That time of year when everyone makes predictions about upcoming trends. We’re willing to bet that, unless they’re from Baba Vanga, most predictions should be taken with a massive grain of salt. Indeed, based on our survey of 2018 prognosticating, most prediction lists you will encounter are at best 75% HOPE and 25% DATA.
Which got us to thinking.
What’s the state of the art these days in the business of predicting? Who is harnessing the waves of data available online to create actionable intelligence?And—because, let’s face it, thinking time, like other time, is money—how might we use our own brand of spaghetti-at-the-wall testing capability to validate predictions?

Homer Simpson FTW

First of all, we wanted to see if any predictions about 2017 came true. Strangely, what rose to the top was The Simpsons’ uncanny 19-year-old prediction that Disney would buy Fox. It wasn’t exactly what we had in mind, but it crystallized our feeling that 2017 was a year in which things that may have once seemed absurd became completely normal.
But we were looking for data-driven predictions. If our Google search results are any indication, it was a tough year for public pronouncements about the future: we didn’t find anyone crowing about their predicting prowess. At best, there were a few people giving themselves high-fives for namby-pamby forecasts like “Prediction #2: Innovation Around Micro-Learning” that, frankly, didn’t even meet our criteria for a real prediction. And psychics are still a thing. We moved on.


Who cares most about predictions? People who can make a lot of money from them. So we checked out what was happening on Wall Street.
In a nutshell, the use of big data-driven analytics appears to be pervasive among most companies that buy or sell securities. As investors have shifted money away from hedge funds and into passive investments like exchange-traded funds that track the market, the pressure is on to compete by having better data. Quant funds—funds that use software and a lot of quantitative inputs to manage money—are hot. And they need an edge.
The action appears to be in so-called alternative data, data from social media, foot traffic in retail stores, and blog posts that can provide early indications of trends. That’s right: your tweets, trips to Walmart, and credit card transactions are all fodder for Wall Street. And as you might imagine, all that activity is creating some legal grey areas.

And The Winner Is...

Who else cares about predictions? Political candidates. When compared with the cutting-edge predictive analytics being developed for Wall Street, election polling seems downright mundane. This four-minute podcast, an interview with polling pundit Professor Spencer Kimball sums up polling methodologies, which range from “not great” to “meh,” as near as we can tell. White space, anyone?