Author: Jude Mc Corry Head of Business Development at The Data Lab
Week two of my new Job as Head of Business Development at the Data Lab and I was “shipped” off to Boston to attend the Big Data Conference and also attend meetings with MIT Sloan, The Innovation Institute at Masstech and Massachuesetts Green High Computing Centre.
It was a really valuable week for me both from a networking and educational perspective where I met lots of interesting people via Mike Cwalinski at Scottish Development International’s Boston office and I was immersed in Hadoop, analytics, machine learning, machine-to-machine, data warehouse, NoSQL, data science, visualization, data management – you name it – I heard it!
I really enjoyed hearing all the data stories” from the Big Data users in the US like Airbnb, Boston Red Sox, US Dept of Commerce, UPS and;
Ebay on Data Innovations
Ashok Ramani, Product Development for DATA at Ebay says they are using analytics to help it understand its customers better. Ambition is to take the kind of personalisation possible in a small shop and apply it to the world of eBay. “In a small store, engaging the customer is key, helping them with search and recommendations, understanding their preferences and learning from existing customers,” he said.
Web metrics data is the raw material Ebay has at his disposal. The auction site generates a huge amount of web analytics, which Ramani described as “the customer journey data”. This tells him what people do on eBay and how they use the site.
The web can offer the same experience [as a local shop], and provide customers with comparisons, so Ebay can learn customers intentions. All this insight drives technology changes at eBay.
#UBERDATA
Silvanus Lee took us through Uber’s business model which is based on the very Big Data principle of crowd sourcing. Anyone with a car who is willing to help someone get to where they want to go can offer to help get them there.
Uber holds a vast database of drivers in all of the cities it covers, so when a passenger asks for a ride, they can instantly match you with the most suitable drivers.
Fares are calculated automatically, using GPS, street data and the company’s own algorithms which make adjustments based on the time that the journey is likely to take. This is a crucial difference from regular taxi services because customers are charged for the time the journey takes, not the distance covered.
These algorithms monitor traffic conditions and journey times in real-time, meaning prices can be adjusted as demand for rides changes, and traffic conditions mean journeys are likely to take longer. This encourages more drivers to get behind the wheel when they are needed and stay at home when demand is low. The company has applied for a patent on this method of Big Data-informed pricing, which is called surge pricing”.
I also heard some fantastic case studies from Suppliers like IBM, Dell, Adobe, Pepper Data, Datameer,etc.; including:
Customer Insights at Netflix
According to Netflix Data should be accessible, easy to discover, and easy to process for everyone, whether your dataset is large or small, being able to visualise it makes it easier to explain. The longer you need to find the data, the less valuable it becomes. This explains why Netflix is the leading Visual Organisation, because at the heart of its business lie some of the most sophisticated Big Data Tools on the planet.
Look at the covers of House of Cards and the 2010 version of Macbeth:
At first glance, they are similar. They both display older” men with blood on their hands – Kevin Spacey and Patrick Stewart, respectively – against primarily black backgrounds. The covers of the two shows are much more similar than dissimilar. At the same time, though, subtle differences exist – and Netflix can precisely quantify those differences also Netflix can see if they have any impact on subscriber viewing habits, recommendations, ratings etc.
Are certain customers trending toward specific types of covers? If so, should personalised recommendations automatically change? What colours appeal to customers, is there an ideal cover for an original series, or should different colours be used for different audiences? This really resonates with me as I am a very visual person and when True Detective and Game of Thrones trailers were first aired they really turned me off watching the series (Yes, I have heard what I am missing, and I totally understand if you Game of Throne junkies have already labelled me as boring!).
Big Data in law enforcement
There was another interesting case study where LA and Santa Clara police department have taken an algorithm used to predict earthquakes, tweaked it and started feeding it crime data. The software can predict where crimes are likely to occur down to 500sqft. In LA there has been a reduction of 33% of burglaries and a 21% reduction in violent crimes where this software is being used.
From speaking to suppliers and end users at the conference and also at my meetings around the Boston area the same issues and opportunities are felt both sides of the Atlantic:
- Big Data is here to stay, but will get bigger with huge opportunities for vendors to innovate, collaborate and help organisations understand and use their data better.
- Data matters across all industry’s no sector is King!
- By 2018 the US alone could face a shortage of between 140,000 and 190,000 people with analytical skills.
From speaking with suppliers, end users and academia in Boston it feels like what we are doing with Data in Scotland here at The Data Lab is exciting, timely, effective, leading edge, and hopefully there is opportunity for lots of collaboration with these organisations and many more to look at Data on world wide basis.
PS: Boston is a great city !!