Machine Learning Misses the Mark

Having moved into a new house recently, I've had some shopping to do. No matter how much we brought along from the old place, inevitably getting settled involves buying more stuff to fill in the gaps. These items range from the trivial (light bulbs & trash bags) to the significant (new home theater kit - woohoo!). Living in a small town and working online all day makes it natural to search Amazon and eBay for the things we need. As I shop, I'm quite conscious of the digital profile being built about me and my proclivities. This doesn't bother me particularly, but it's interesting to see how (in)effectively the information is being utilized to market other offers to me. 

Based on what I see, these companies are still missing the mark more often than not. Given their business model and renown for leading the way in data science and machine learning, it's rather surprising to see the results. For example, on eBay I bought a Subzero refrigerator and a Viking range, and although I'm very pleased with these items, it is weird that eBay now thinks I'm most likely to buy more fridges and stoves. The basic logic that these are almost always infrequent purchases is somehow lost on their analytics models. Same problem on Amazon, where I bought a projector for the theater, and now the algorithm thinks I'm going to want to get another projector or perhaps a big screen TV.

Amazon_plugsAmazon gets this logic wrong in other ways, too. Since I recently bought volume two of my favorite comic book, ahem, graphic novel, the landing page is now recommending I pick up volume one of the series. It's obvious to any human that very few shoppers will read books in reverse order. Similarly, while I bought a missing power cable for a computer, this isn't the exciting, "must have" kind of item that should be prominently displayed in a dozen different variations the next time I visit the website. Just ain't going to make me want to buy more. It was a one-off necessity, not a candidate for repeated impulse buys.

Yet companies like Amazon and eBay have clearly devoted significant investment and effort to solving the question of "what will he likely buy next?" and they have done this with machine learning. Shoot, Amazon even offers machine learning tools in AWS. Why are we still so far off? Maybe because it's hard to do; maybe it's because as a consumer, I have high expectations. Yet when I walk into the local appliance store, the salesman there immediately recognizes me and intuitively knows I'm not going to buy two new kitchen stoves in the same month. If all the vaunted capabilities of a company like Amazon, a company essentially built around big data, still gets it wrong, what hope is there for a smaller organization without the same resources?

The good news is that many other players are working on this space, and making it easier to leverage their solutions. These include notables like Microsoft with Azure ML, IBM with Watson and Spark, Databricks with Spark, Skytree, Dato, Hadoop distribution vendors, and others. I hope to see better results soon. 

big data analysis

Topics: Data Platforms, Analytics, & AI